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Abstract 

Air  Force  (AF)  Concept  Development  and  Experimentation  (CD&E)  continually 
progresses  the  evolution  of  AF  while  achieving  national  security  and  military  objectives. 
CD&E  experiments  on  future  challenges,  procurement  of  new  weapon  systems  and  tests 
existing/innovative  strategies  as  potential  solutions.  The  primary  tool  that  CD&E  utilizes 
in  conducting  these  experiments  is  wargaming.  This  thesis  provides  a  foundation  to 
incorporate  logistics  into  Air  Force  Title  10  wargames.  More  specifically,  we  capture  Air 
Force  Materiel  Command’s  (AFMC)  Agile  Combat  Support  (ACS)  within  an  unclassified 
general  wargame  scenario.  Logistics  has  been  omitted  from  wargames  for  a  multitude  of 
reasons  throughout  the  years.  We  develop  a  logistics  simulation  model  of  a  simplified 
wargame  scenario  designed  to  be  run  within  the  Logistics  Composite  Model  (LCOM) 
Analysis  Toolkit  (ATK)  version  4.0  before  a  wargame  initiates.  We  capture  ACS  within 
the  stochastic  simulation  by  incorporating  engine  failures,  maintenance  crews, 
ammunition,  fuel,  and  various  other  logistics  metrics.  By  varying  the  types  of  sortie 
operations  and  the  logistics  support  available,  further  insight  is  gathered  on  Blue  Force 
capabilities.  We  develop  decision  quality  information  to  present  to  a  decision  maker  by 
combining  statistical  and  multivariate  analysis.  Our  approach  showcases  how  to  gather 
insights  from  ACS  metrics,  including  development  of  a  metamodel  using  only  four 
metrics  to  successfully  predict  key  ACS  Measures  of  Effectiveness  (MOEs).  Ultimately, 
we  design,  analyze  and  demonstrate  that  logistics  can  and  should  be  incorporated  into 
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wargames. 


AFIT-ENS-MS-15-M-127 


To  Mom  and  Dad 


IX 


Acknowledgments 


I  would  like  to  express  my  sincere  appreciation  to  my  faculty  advisor,  Dr.  J.O.  Miller,  for 
his  guidance  and  support  throughout  the  course  of  this  thesis  effort.  I  would,  also,  like  to 
thank  Dr.  D.K.  Ahner,  for  his  invaluable  insight  throughout  our  research. 


Daniel  A.  Krievs 


x 


Table  of  Contents 


Page 

Abstract . viii 

Table  of  Contents . xi 

List  of  Figures . xiii 

List  of  Tables . xiv 

I.  Introduction . 1 

Problem  Statement . 1 

General  Issue . 2 

Research  Objectives/Questions/Hypotheses . 3 

Scope . 3 

Methodology . 4 

Summary . 5 

II.  Literature  Review . 6 

Wargame  Overview . 6 

USAF  Wargames . 9 

Logistics  in  Wargames . 10 

Agile  Combat  Support  (ACS) . 12 

Logistics  Composite  (LCOM)  Simulation  Model . 13 

Summary . 16 

III.  Methodology . 17 

Wargame  Overview . 1 7 

LCOM-ATK  Overview . 19 

Logistics  Simulation . 25 

Analysis  Methodology . 31 

xi 


Conclusion 


32 


IV.  Analysis  &  Results . 34 

Statistical  Analysis . 35 

Multivariate  Analysis . 39 

Summary . 53 

V.  Conclusions  and  Recommendations . 54 

Conclusions  of  Research . 54 

Recommendations  for  Future  Research . 58 

Appendix  A:  Measures  of  Effectiveness . 60 

Appendix  B:  Multivariate  Analysis  Code . 63 

Appendix  C:  Quad  Chart . 69 

Bibliography . 70 


xii 


List  of  Figures 


Page 

Figure  1.  LCOM  Process  (Erdman,  2014:27) . 21 

Figure  2.  Wargame  Scenario . 30 

Figure  3.  Simulation  Model  (Erdman,  2014:5) . 3 1 

Figure  4.  Sortie  Completion  Rate . 35 

Figure  5.  JSF  Unscheduled  Maintenance  Rate . 36 

Figure  6.  PCA  Horn’s  Test . 42 

Figure  7.  PCI  vs.  PC9 . 44 

Figure  8.  PCI  vs.  PC9  Validation . 45 

Figure  9.  DA  Model  Efficiency . 46 

Figure  10.  Factor  3  vs.  Factor  10 . 48 

Figure  11.  Factor  3  vs.  Factor  10  Validation . 49 

Figure  12.  ANN  Signal  to  Noise  Ratio . 50 

Figure  13.  ANN  Confusion  Matrix . 51 


xiii 


List  of  Tables 


Page 

Table  1.  Group  A  (Walker  et  al.,  2010) . 22 

Table  2.  Group  B  (Walker  et  al.,  2010) . 22 

Table  3.  Group  C  (Walker  et  al.,  2010) . 23 

Table  4.  Group  D  (Walker  et  al.,  2010) . 23 

Table  5.  Group  E  (Walker  et  al.,  2010) . 24 

Table  6.  Group  F  (Walker  et  al.,  2010) . 24 

Table  7.  Group  G  (Walker  et  al.,  2010) . 25 

Table  8.  Group  H  (Walker  et  al.,  2010) . 25 

Table  9.  Logistics  Simulation  Analysis . 37 

Table  10.  PC  A  Variance  Explained . 43 

Table  11.  FA  Loadings  Matrix . 48 


xiv 


INTEGRATING  AGILE  COMBAT  SUPPORT  WITHIN  TITLE  10  WARGAMES 


I.  Introduction 

This  thesis  provides  an  approach  of  capturing  logistics  within  wargames. 
Wargaming  has  been  used  throughout  the  history  of  warfare  to  ensure  battle  commanders 
are  prepared  and  informed  of  potential  scenarios  they  might  encounter.  As  time  has 
progressed,  the  warfare  has  changed  from  tribal  to  medieval  to  trench  to  guerilla  and  now 
to  modern  warfare.  As  these  types  of  warfare  have  evolved,  the  form  of  wargames  has 
had  to  adapt  ranging  from  table  top  discussion  to  combat  model  simulations.  While  the 
types  of  simulations  used  for  wargames  has  adapted,  there  is  still  a  typical  flaw  in 
omitting  logistics  from  the  models.  The  reasoning  has  been  from  a  culmination  of  things, 
but  the  bottom  line  is  with  today’s  budgetary  constraints,  battle  commanders  must  be 
cognizant  of  the  supplies  that  they  require  to  complete  missions  and  maintain  the  United 
States  Air  Force’s  (USAF)  dominance  in  air,  space,  and  cyberspace. 

Problem  Statement 

President  Obama  signed  the  “Priorities  for  the  21st  Century  Defense”  on  January 
3,  2012.  The  memorandum  states  that  there  will  be  a  rebalancing  of  US  troops  toward 
the  Asia-Pacific  region.  Air  Force  Materiel  Command  (AFMC/A4&A8)  then  conducted 
a  study  to  detennine  how  the  Air  Force  (AF)  assesses  its  capabilities  in  future  air 
campaigns.  The  results  were  astonishing  because  Agile  Combat  Support  (ACS)  is  only 
accounted  for  30  days  into  the  future  and  are  omitted  from  studies  that  assess  long  term 
anned  conflicts  such  as  wargames.  These  results  were  not  pleasing  and  something 
needed  to  be  done  to  fix  the  gap  between  wargames  and  logistics.  This  thesis  effort  is  to 
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determine  how  ACS  can  be  captured  effectively  and  timely  in  Title  10  wargames.  These 
wargames  are  referred  to  as  Title  10  due  to  the  service  chiefs’  responsibilities  outlined  in 
U.S.  Code  Title  10  to  train  their  respective  forces. 

General  Issue 

Logistics  has  been  continuously  omitted  from  wargames  for  many  reasons. 
Traditionally,  logistics  has  been  an  afterthought  because  of  the  ability  to  continuously 
increase  budgets  to  support  anned  conflicts.  As  time  is  progressing,  the  Department  of 
Defense  (DoD)  budget  is  increasingly  more  constrained  with  a  growing  emphasis  on 
accurately  projecting  the  costs  associated  with  future  engagements.  Wargaming  has  been 
the  hallmark  of  US  forces  in  determining  how  the  US  will  overcome  opponents.  The 
outcomes  of  these  wargames  highly  influence  location  of  personnel/equipment, 
procurement  of  weapon  systems,  and  defining  the  tactics  needed  to  be  successful; 
however,  not  incorporating  logistics  is  a  failure  that  needs  to  be  addressed. 

In  recent  years,  computer  simulations  for  wargames  have  been  trying  to  capture 
logistics  within  their  combat  models.  The  simulations  have  failed  time  and  time  again  for 
many  reasons.  One  of  the  main  issues  of  incorporating  logistics  into  wargames  is  the 
conflicting  aggregation  levels  of  the  models.  Wargame  combat  models  have  primarily 
been  used  at  the  campaign  level,  while  logistics  simulation  models  have  used  a  more 
detailed  tactical  level  model  without  explicit  modeling  many  combat  operations. 

Wargame  combat  models  have  stayed  at  the  campaign  level  because  of  the 
following: 

Time  saved  by  running  a  more  aggregated  model 

The  focus  of  wargames  being  on  opponent  decision  making 
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Commanders  being  overwhelmed  by  data  from  a  lower  level  model 
Logistic  models  have  been  unable  to  run  during  wargames  because  of  the  short 
window  of  time  to  complete  the  model.  Wargames  are  completed  on  24  hour  cycles,  but 
this  short  period  in  time  makes  it  difficult  to  capture  reasonable  logistic  impacts. 


Research  Obj  ectives/Questions/Hypotheses 

The  goal  of  this  study  is  to  answer  the  following  questions: 

1 .  What  simulation  platfonn  is  most  appropriate  for  incorporating  logistics 
into  wargames? 

2.  At  what  point  should  the  logistics  simulation  being  completed  within  the 
wargame  process? 

3.  Is  there  a  logistics  metamodel  that  can  be  adjusted  and  completed  during 
wargames? 

4.  What  metrics  are  the  most  vital  and  able  to  represent  ACS? 

Our  goal  is  to  produce  a  proof  of  concept  logistics  simulation  within  a  wargame 
scenario  to  model  the  constraints  and  capabilities  associated  with  ACS.  The  results  from 
this  simulation  are  then  garnered  to  develop  further  limitations  and  constraints  regarding 
the  capabilities  of  US  forces.  We  believe  that  the  most  ideal  approach  in  capturing 
logistics  within  wargames  is  to  complete  a  logistics  simulation  before  the  wargames 
begin  and  to  capture  key  logistics  outcomes  in  a  metamodel.  This  provides  wargame 
commanders  with  more  realistic  force  capabilities  including  ACS  during  a  wargame  and 
an  opportunity  to  adjust  their  strategies  based  on  these  updates. 


Scope 


This  research  responds  to  a  question  from  AFMC/A4/A8  regarding  a  method  to 
capture  logistics  effectively  and  timely  in  a  wargame.  The  modeling  of  an  enhanced  or 
deployed  logistics  capability  in  support  of  a  specific  scenario  needs  to  be  broken  down 
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into  two  subcomponents.  One  component  being  the  delivery  of  initial  parts  and 
equipment  to  support  ACS  in  the  wargame  carried  out  by  Air  Mobility  Command 
(AMC).  The  other  component  is  ACS  logistic  operations  being  conducted  during  the 
wargame  handled  by  AFMC.  The  primary  objective  of  our  research  is  to  identify  feasible 
metrics  to  capture  ACS  within  a  wargame  and  to  present  these  metrics  as  part  of  military 
decision  making  process. 

This  thesis  focuses  on  the  computer  modeling  of  ACS  in  Title  10  wargames.  We 
work  under  the  confinements  and  goals  associated  with  United  Engagements  (UE).  We 
are  not  attempting  to  replace  or  construct  a  theory  of  war.  Rather  this  thesis  attempts  to 
use  the  ideas  of  previous  wargames  and  answer  its  basic  question  of  how  a  strategic 
wargame  can  capture  logistics. 

Methodology 

We  first  develop  a  mission  statement  for  logistics  being  incorporated  into 
wargames.  This  statement  drives  the  analysis  and  detennination  along  the  course  of  the 
study.  After  this  we  move  into  understanding  why  logistics  has  been  omitted  from 
wargames.  This  cause  leads  into  what  type  of  simulation  platfonn  to  utilize.  We  develop 
a  fictional  wargame  scenario  around  this  simulation  model  and  develop  a  proof  of 
concept.  The  analysis  is  partitioned  into  two  cases.  The  first  case  analyzes  the  entire  set 
of  output  data  from  the  simulation  to  develop  constraints,  limitations,  and  provide  a  more 
accurate  representation  of  the  ACS  capabilities  and  constraints  for  forces  involved  in  the 
wargame.  The  second  case  revolves  around  the  idea  of  developing  a  logistics  metamodel 
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that  can  be  run  during  an  actual  wargame.  Both  of  these  cases  focus  on  the  goal  of 
incorporating  logistics  into  wargames. 

Summary 

This  thesis  explores  the  realm  of  the  relationship  between  logistics  and  wargames. 
We  discuss  why  there  has  been  a  lack  of  importance  to  incorporate  logistics  and  what 
issues  arise  when  wargames  attempt  to  bring  in  logistics.  We  focus  specifically  on  ACS 
and  develop  a  proof  of  concept  that  develops  a  further  understanding  of  how  to  capture 
ACS  capabilities  and  constraints  for  our  forces  included  in  the  wargame.  A  combat 
mission  doesn’t  just  require  a  squadron  of  fighter  jets;  it  also  requires  the  personnel  and 
equipment  to  support  them.  The  next  chapter  of  this  thesis  provides  an  overview  of 
wargames  and  the  lack  of  connection  between  logistics  and  wargames. 
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II.  Literature  Review 


The  purpose  of  this  chapter  is  to  review  literature  on  wargames,  the  issues 
associated  with  incorporating  logistics  into  wargames,  Agile  Combat  Support  (ACS),  and 
the  Logistics  Composite  (LCOM)  simulation  model.  We  begin  the  chapter  by  discussing 
the  applications  of  wargames  and  how  the  USAF  utilizes  wargames.  We  then  investigate 
the  underlying  issues  with  capturing  logistics  in  wargames.  Finally,  we  discuss  how  ACS 
is  a  part  of  AF  logistics  and  why  LCOM  is  the  appropriate  platform  to  complete  a 
logistics  model  for  a  wargame. 

Wargame  Overview 

Wargaming  has  been  used  throughout  history  as  a  fundamental  tool  in  developing 
military  strategy.  These  strategies  have  then  been  incorporated  into  training,  education, 
procurement,  and  many  other  areas  of  the  US  military  (Perla,  1990:3).  The  USAF 
defines  wargames  as  “a  simulation,  by  whatever  means,  of  a  military  operation  involving 
two  or  more  opposing  forces,  using  rules,  data,  and  procedures  designed  to  depict  an 
actual  or  assumed  real  life  situation”  (McHugh  and  Fischer,  1966:9).  The  goal  of 
wargames  is  ultimately  communication  (Perla,  1990: 185).  This  communication 
showcases  the  cause  and  effect  of  wargame  weapon  systems  interactions  and  the 
decisions  made  by  the  commanders. 

This  decision  making  is  what  sets  apart  a  simulation  from  a  wargame. 
Incorporating  commanders  as  decision  makers  into  wargame  models  allows  for  unique 
unrepeatable  simulations.  Thus  wargames  are  not  regarded  as  analytical  tools  (Long, 
1993:5);  however,  this  is  the  most  essential  part  of  wargames.  The  human  in  the  loop 
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allows  wargames  to  be  unpredictable  and  deviate  from  the  mundane  rules  and  algorithms 
utilized  by  computer  simulations.  “Good  wargames  must  be  structured  to  help  human 
players  make  decisions  and  allow  them  to  learn  about  the  effects  of  those  decisions” 
(Perla,  1990:164).  Ultimately,  this  human  element  of  decision  making  is  the  key  in 
providing  the  educational  experience  gathered  from  wargames. 

The  educational  experience  is  the  ability  of  wargames  to  allow  commanders  to 
test  warfare  strategy  against  enemy  opponents  without  having  loss  of  life.  This  asset 
increases  the  wargame  commanders’  knowledge  of  war  and  increases  their  decision 
making  abilities.  Wargames  are  able  to  achieve  this  because  they  use  mathematical 
combat  models  to  simulate  the  physical  interactions  between  weapon  systems  (Perla, 
1990:164).  These  models  are  developed  to  understand  the  Measures  of  Effectiveness 
(MOEs)  of  one  weapon  system  against  an  enemy  force.  Wargame  commanders  then 
theorize  and  provide  their  decisions  of  force  movement  to  the  combat  models.  In 
addition,  opposing  commanders  are  using  similar  weapon  systems  and  strategizing  to 
achieve  their  missions  at  the  other’s  expense.  This  becomes  an  issue  in  today’s 
environment  where  USAF  opponents  are  not  at  the  same  level  of  technology  and  the 
decisions  that  the  opponents  make  are  far  different  then  gathered  from  the  wargame. 
Models  that  can  accurately  capture  the  actual  opponent’s  supply,  weapon  systems,  and 
decision  making  provide  far  more  insight  then  playing  a  wargame  against  ourselves. 

Wargames  have  evolved  overtime  to  minimize  this  issue  of  capturing  an 
opponent’s  decision  making  by  forcing  opposing  commanders  to  study  and  become 
familiar  with  enemy  tactics  (Perla,  1990:203).  Their  primary  mission  is  to  become 
knowledgeable  in  the  enemy’s  culture  to  develop  a  mindset  that  is  the  foundation  to  their 
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judgment  during  wargames.  Opposing  commanders’  expertise  in  their  respective  enemy 
culture  provides  the  necessary  realism  against  the  friendly  forces,  but  it  fails  to  provide  a 
justification  as  to  why  an  enemy  force  moves  to  a  particular  location  or  engages  in  an 
attack.  The  issues  that  arise  from  here  derive  from  the  aggregation  level  of  wargames. 

Dr.  C.  L.  Helwig  brought  the  concept  of  aggregation  to  light  back  in  1780  when 
he  “employed  a  single  playing  piece  to  represent  a  large  body  of  soldiers”  (Perla 
1990:18).  Wargames  follow  Dr.  Helwig’s  idea  by  having  their  combat  models  operate  at 
the  strategic  aggregation  level.  At  this  level,  like  forces  are  gathered  together  to  form 
wings,  battalions,  etc.  There  isn’t  individual  weapon  system  versus  weapon  system  as  in 
tactical  combat  models,  but  instead  a  large  force  opposing  another  large  force.  The 
inputs  for  the  model  are  things  to  the  effect  of  defeated  army  battalions  and  the  outputs 
are  surrendering.  The  strategic  combat  model  then  uses  computer  algorithms  to  convert 
these  inputs  into  outputs  by  using  mechanisms  that  are  not  only  militaristic  but  political, 
psychological,  diplomatic,  etc.  A  great  example  as  to  why  strategic  combat  models  don’t 
only  include  military  operations  (e.g.  destroying  infrastructure)  is  the  recent  war  in 
Afghanistan.  US  forces  have  destroyed  a  large  amount  of  infrastructure  and  controlled 
the  land  within  the  country  in  a  short  amount  of  time,  yet  the  war  continued  for  a  decade. 
This  example  showcases  how  wars  are  not  always  symmetric  and  easily  understood. 
Winning  a  war  sometimes  takes  more  than  destroying  enemy  forces  and  that’s  a  reason 
why  wargames  are  conducted  at  the  strategic  level.  Strategic  combat  models  have  the 
ability  to  incorporate  symmetric  and  asymmetric  effects.  The  downside  is  that  there  is  a 
vast  amount  of  assumptions  when  running  at  the  strategic  level.  These  assumptions  take 
logistics  for  granted  and  assume  that  a  forward  operating  base  (FOB)  maintains,  operates, 
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and  supports  any  mission  used  during  the  wargame.  The  example  of  the  “War  on  Terror” 
in  Afghanistan  is  a  perfect  reason  why  logistics  shouldn’t  be  assumed.  This  type  of  war 
refocused  from  force  on  force  attrition  to  prolonged  forward  sustainment.  The  questions 
changed  from  how  well  our  airpower  can  compete  against  the  enemy  forces  to  questions 
like  the  following: 

What  is  the  minimal  amount  of  aircraft  in  the  forward  operating  bases 
needed  to  maintain  day  to  day  operations? 

Can  the  USAF  continue  to  fund  weapon  system  upgrades  while 
maintaining  overseas  operations? 

What  is  the  cost/benefit  of  operating  a  FOB  as  opposed  to  long  range 
weapon  systems? 

These  are  only  a  small  snapshot  of  questions  regarding  strategic  level  AF  leaders, 
and  as  with  the  past,  wargames  are  the  likely  tool  to  provide  insights  on  these  questions. 
The  ongoing  evolution  of  wargames  must  continue  and  have  the  ability  to  include  the 
effects  of  logistics.  Incorporating  logistics  will  more  accurately  represent  the  forces 
involved  in  wargames,  but  more  importantly,  allow  wargames  to  be  even  better  tools  for 
answering  questions  in  today’s  world. 

USAF  Wargames 

Wargames  are  used  within  several  levels  of  the  Air  Force.  Each  level  provides 
different  types  of  insight,  but  we  concentrate  on  Title  10  wargames.  “As  the  Goldwater- 
Nichols  Act  of  1986  gave  the  service  chiefs  responsibility  under  U.S.  Code  Title  10  to 
train,  man,  and  equip  their  individual  forces”  (Duchanne,  2012: 1).  The  Air  Force  began 
conducting  Title  10  wargames  in  1995  by  creating  two  types  of  wargames:  Unified 
Engagement  (UE)  and  Future  Capabilities  Game  (Duchanne,  2012:2).  UE  wargames  are 
completed  on  even  years  with  Future  Capabilities  Games  being  held  on  odd  years.  These 
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two  types  of  wargames  have  entirely  different  objectives,  but  we  strictly  focus  on  Unified 
Engagements. 

The  goal  of  UE  is  to  “address  military  challenges  and  concept  exploration”  within 
the  Pacific  and  European  theaters  (Ducharme,  2012:2).  The  wargames  are  conducted  in 
theater  and  are  based  on  technologies  that  are  near  tenn  (approximately  12  years  out). 

UE  is  established  to  be  a  tool  that  an  operational  Commander  can  use  to  develop  and  test 
campaign  level  strategies  in  their  respective  theater  (Duchanne,  2012:43).  There  are 
three  teams  or  forces  involved  with  the  wargames.  The  white  team  adjudicates;  the  Blue 
Force  represents  the  US  forces  while  the  Red  Force  represents  the  enemy  forces  (Caffrey, 
2008:40).  The  players  during  these  wargames  are  the  decision  makers  and  are  using 
history,  culture,  and  doctrine  to  develop  strategies  and  crisis  action  plans  (Caffrey, 
2008:43).  Each  UE  is  different  and  the  rules  for  winning  the  wargame  change;  however, 
the  main  goal  is  not  necessarily  to  win  the  wargame  but  to  come  out  with  useful  insights 
on  operational  strategies  (Haffa,  2001).  Because  of  the  complexity  of  typical  UE 
scenarios,  constructive  computer  simulations  are  used  to  capture  daily  Red  and  Blue 
Force  moves  as  well  as  adjudication  of  battlefield  engagements. 

Logistics  in  Wargames 

There  is  no  definitive  date  in  which  wargames  were  created.  It  is  due  to  the  fact 
that  wargames  have  been  executed  throughout  the  history  of  mankind.  The  definitions 
vary  but  the  heart  of  each  definition  remains  the  same.  Wargames  are  fictional  studies 
carried  out  by  military  leaders  to  construct,  amend  and/or  validate  a  war  plan  (Collins, 
2012).  There  are  various  forms  of  wargames  that  are  also  carried  out  by  the  commercial 
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industry;  however,  we  focus  on  US  Air  Force  wargames  and  understand  the  role  that 
logistics  has  played  in  wargames.  This  section  illustrates  the  shortcomings  that  logistics 
has  had  in  wargames  and  the  numerous  issues  involved  with  these  shortcomings. 

One  of  the  key  issues  of  UE  is  a  failure  to  evaluate  the  logistic  operations  (Haffa, 
2001).  Logistics  is  regarded  as  an  operational  constraint  and  fails  to  incorporate  the 
“logistic  support  effects  on  campaign  planning”  (LaPlante  et  ah,  1996:97).  Blue  Forces 
overestimate  their  effectiveness  level  and  completely  disregard  the  logistics  involved  in 
supporting  their  campaign.  For  example,  the  kill  rates  included  in  the  simulations  require 
that  munitions  be  provided  for  each  of  the  weapon  systems;  however,  the  amount  of 
ammunition  available  is  considered  unlimited  and  use  is  not  tracked  (Haffa  2001).  This 
in  turn  over  predicts  the  effectiveness  of  US  forces,  under  predicts  the  red  forces, 
underestimates  the  budget  and  provides  less  insightful  results  to  a  large  number  of 
policies  and  doctrine  (Caffrey,  2008:37). 

Other  reasons  that  logistics  has  failed  to  be  captured  in  wargames  are  because 
logisticians/warfighters  were  excluded  from  the  wargames  (LaPlante,  1996:97)  and  the 
cost/complexity  of  incorporating  logistics  is  far  too  great  (Ducharme,  2012:4).  Over  the 
last  couple  of  decades  logisticians  and  warfighters  have  been  incorporated  into 
wargames;  however,  there  were  still  issues  in  capturing  logistics  in  simulation  models 
(LaPlante,  1996:97). 

The  main  reason  for  this  matter  was  still  the  issues  with  complexity  and  cost  of 
further  developing  the  wargame  models;  but  funding  a  project  to  develop  and  maintain 
logistics  within  a  wargame  simulation  model  is  far  less  expensive  than  underestimating 
the  budget  for  a  war  in  the  Pacific. 
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Agile  Combat  Support  (ACS) 

Logistics  impacts  every  facet  of  the  Air  Force.  From  the  delivery  of  F-35s  to 
helmets  delivered  to  Airmen,  logistics  plays  a  role.  The  AF  breaks  logistics  into  two 
commands:  Air  Mobility  Command  (AMC)  and  Air  Force  Materiel  Command  (AFMC). 
This  paper  focuses  on  the  AFMC  piece.  AFMC  provides  logistics  by  executing  Agile 
Combat  Support  (ACS).  “ACS  is  the  ability  to  create,  protect,  and  sustain  air  and  space 
forces  across  the  full  range  of  military  operations”  (Westhauser,  2011: 1).  It  is  the  Air 
Force’s  capability  that  is  responsible  for  detennining  what  is  deployed  and  how  it  will 
arrive  and  return  to  CONUS  safely. 

The  Air  Force  Doctrine  Document  4-0  Combat  Support  defines  ACS  using  the 
following  attributes: 

Agility:  ensuring  timely  deployment  concentration,  adaptive  employment 
and  resourceful  sustainment  of  air  and  space  power  (Westhauser,  2011: 1). 
Reliability:  competency  and  health  of  personnel,  dependability  of 
equipment,  and  trustworthiness  of  information  (Westhauser,  2011 :2). 
Integration:  incorporate  diverse  parts  into  a  common  team  to  create  a 
synergistic  effect  (Westhauser,  201 1:2). 

Responsiveness:  ACS  capabilities  are  the  right  size,  correct  time  and 
location  (Westhauser,  2011 :2). 

ACS  effects  are  measured  by  readied  forces,  prepared  battlespace,  positioned 
forces,  employed  forces,  sustained  forces,  and  recovered  forces  (Westhauser,  201 1:3). 
These  metrics  are  ultimately  provided  to  the  Commander  Air  Force  Forces 
(COMAFFOR).  ACS  then  uses  its  master  processes  to  apply  the  attributes  and  produce 
the  desired  effects  set  forth  by  the  COMAFFOR.  ACS  maintains,  supplies,  and 
distributes  at  “operating  locations  to  achieve  the  mission  and  assure  the  operational  utility 
of  all  personnel,  materiel,  equipment,  and  the  operating  location  infrastructure” 
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(Westhauser,  201 1:6).  The  services  that  ACS  provides  include  lodging,  medical, 
religious,  postal,  maintenance,  and  many  others  (Westhauser,  201 1:7). 

ACS  is  a  part  of  every  AF  functional  area  and  is  a  vital  component  in  the  Air 
Force’s  mission  to  train,  equip,  and  employ  air  and  space  power.  Omitting  their  role 
during  wargames  by  assumptions  is  not  a  good  practice. 

Logistics  Composite  (LCOM)  Simulation  Model 

The  purpose  of  this  section  is  to  provide  the  readers  with  what  the  Logistics 
Composite  (LCOM)  simulation  model  is  and  why  we  feel  that  this  is  an  appropriate 
model  to  use  for  our  wargame. 

LCOM  is  a  discrete  event  simulation  that  uses  distributions  and  random  number 
generators  to  capture  maintenance  and  optimize  manpower  levels  (Cole,  2007).  The 
LCOM  model  is  a  composite  of  modules,  written  in  SIMSCRIPT  II,  which  communicate 
with  each  other  to  function  as  a  cohesive  entity  (AFLCMC/EZJS,  2013).  The  three 
modules  are:  the  Input  Module  that  preprocesses  the  data,  the  Main  Module  that  runs  the 
simulation,  and  the  Post  Processor  Module  that  analyzes  post  simulation  data 
(AFLCMC/EZJS,  2013).  The  software  is  used  to  simulate  studies  concerning  Air  Force 
base  level  functions  (e.g.  maintenance  and  supply)  and  manpower  studies.  This  in  turn, 
allows  operators  and  strategic  leaders  to  assess  the  availability  of  support  resources  and 
operational  weapon  systems  (AFLCMC/EZJS,  2013). 

The  original  LCOM  was  created  in  1966  by  the  Air  Force  Logistics  Command 
and  the  Rand  Corporation  (AFLCMC/EZJS,  2013).  LCOM  was  initially  utilized  by 
organizations  like  the  Tactical  Air  Command,  the  Air  Force  Human  Resource  Lab,  and 
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the  Air  Force  Systems  Command  for  various  studies  (Erdman,  2014:6).  The  results  of 
these  studies  provided  insights  to  weapon  system  procurement  and  Air  Force  aircraft 
maintenance  manning  standards.  LCOM  significantly  aided  in  exploring  the  trade-offs 
between  various  types  of  assets  used  for  weapon  systems  throughout  the  acquisition  stage 
(AFLCMC/EZJS,  2013).  The  impacts  of  these  studies  were  a  key  contributor  to  the 
overall  reduction  in  weapon  system  costs  (AFLCMC/EZJS,  2013). 

There  have  been  an  abundant  amount  of  upgrades  since  its  creation  in  1966. 
Currently  the  Air  Force  Life  Cycle  Management  Center  (AFLCMC)  operates  and 
maintains  LCOM.  Their  most  recent  update  of  the  model  was  released  in  2014  and 
known  as  LCOM-ATK  (AFLCMC/EZJS,  2013).  While  the  interfaces  have  changed,  the 
goal  and  applicability  of  LCOM  remains  the  same.  LCOM  is  meant  to  capture  logistics 
and  provide  insights  of  maintenance  and  supply  to  Air  Force  leaders  (AFLCMC/EZJS, 
2013). 

One  of  the  great  things  about  this  software  is  that  it  leaves  a  significant  amount  of 
control  over  the  level  of  detail  that  the  model  environment  possesses.  This  was  vital  in 
our  research  because  the  conflicting  aggregation  levels  of  wargames  and  logistic  models. 
This  was  the  key  in  choosing  a  logistics  model  as  our  base  model  as  opposed  to  a 
wargame  combat  model.  The  LCOM  model  is  designed  to  simulate  part  failures,  or  other 
similar  subsystems,  and  process  the  spares  to  capture  the  support  system  during  an 
engagement  over  a  given  period  of  time  (AFLCMC/EZJS,  2013).  This  allows  its  users  to 
tailor  the  model  into  a  plethora  of  different  types  of  logistics. 

The  Input  Module  decreases  and  reformats  the  data  provided  by  users  into  a  data 
structure  appropriate  for  the  Main  Module  (AFLCMC/EZJS,  2013).  More  specifically, 
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the  Input  Module  gathers  the  parameters  set  by  the  user,  constructs  a  flying  program 
based  in  terms  of  sortie  operations  and/or  activities  requiring  specific  types  of  aircraft  or 
non-aircraft  resources  (AFLCMC/EZJS,  2013).  The  Input  Module  simply  gathers  all  the 
data  created  by  the  user,  generates  sorties  and  creates  the  corresponding  environment 
used  in  the  Main  Module. 

The  Main  Module  carries  out  the  simulations  within  the  environment  created  by 
the  user  and  the  Input  Module  (AFLCMC/EZJS,  2013).  The  Main  Module  process  is  as 
follows:  it  begins  with  the  sortie  schedule,  incorporates  part  failures/malfunctions  based 
on  validated  and  historical  data,  gathers  necessary  resources  to  complete  the  scheduled 
sortie,  allots  the  time  required  in  making  scheduled  or  unscheduled  repairs,  and 
incorporates  the  exchanges  of  the  demand  process  for  resources  (AFLCMC/EZJS,  2013). 
Each  of  these  described  responses  is  only  a  small  portion  of  what  LCOM  truly  does.  The 
ability  to  describe  the  maintenance  and  support  of  aircrafts  over  multiple  days,  months  or 
even  years  is  a  key  factor  on  why  the  model  has  lasted  and  continues  to  thrive  in  today’s 
every  growing  world. 

The  Post  Processor  Module  provides  users  with  the  opportunity  to  conduct  post¬ 
simulation  analysis  on  the  data  gathered  from  the  Main  Module  (AFLCMC/EZJS,  2013). 
The  data  gathered  from  the  Main  Module  is  large  and  robust  because  it  covers  an 
extended  period  in  time  and  tracks  a  large  volume  of  entities.  The  Post  Processor  Module 
consolidates  all  of  these  data  points  into  various  types  of  reports,  and  develops  visual 
representations  to  provide  further  insights  to  its  users  (AFLCMC/EZJS,  2013). 

The  output  of  LCOM-ATK  is  consolidated  into  reports  (AFLCMC/EZJS,  2013). 
The  reports  are  groups  of  similar  statistics  or  MOEs  collected.  For  example,  statistics 
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gathered  from  aircraft  activities  are  gathered  into  a  single  report  referred  to  as  Group  C 
(aircraft).  The  reports  are  gathered  in  Group  A  (mission).  Group  B  (activity),  Group  C 
(aircraft),  Group  D  (personnel),  Group  E  (shop-repair),  Group  F  (supply),  Group  G 
(equipment)  and  Group  H  (facility)  (Erdman,  2014:28).  A  complete  listing  of  the 
measures  used  for  our  logistic  simulation  is  included  in  Appendix  A  of  this  thesis. 

Summary 

This  chapter  discussed  the  history  of  wargames  and  the  issues  associated  with  not 
incorporating  logistics.  The  heart  of  the  problem  is  the  differences  of  fidelity  between  a 
typical  wargame  and  logistics  simulation.  Wargame  simulations  are  aggregated  at  the 
campaign  level,  while  logistics  simulations  are  constructed  with  a  higher  level  of  detail  at 
the  operational/tactical  level.  This  difference  in  fidelity  has  been  a  key  factor  in 
wargames  being  played  with  the  exclusion  of  logistics.  The  next  chapter  of  this  thesis 
describes  our  methodology  for  incorporating  logistics  within  AF  wargames. 
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III.  Methodology 


Solving  how  logistics  can  be  captured  during  wargames  requires  our  study  to  start 
from  the  beginning  of  wargames.  We  explore  what  wargames  are  and  why  USAF 
perfonns  wargames.  Once  we  understand  the  added  benefits  of  wargames  to  USAF  we 
can  focus  on  why  logistics  should  be  incorporated.  This  understanding  ultimately  helps 
us  shape  our  methodology  to  capture  ACS  capabilities  and  constraints  in  a  wargame. 

Wargame  Overview 

There  is  no  definitive  date  in  which  wargames  were  created.  It  is  due  to  the  fact 
that  wargames  have  been  executed  throughout  the  history  of  mankind.  The  definitions 
vary  but  the  heart  of  each  definition  remains  the  same.  Wargames  are  fictional  studies 
carried  out  by  military  leaders  to  construct,  amend  and/or  validate  a  war  plan  (Collins, 
2012).  There  are  various  fonns  of  wargames  that  are  also  carried  out  by  the  commercial 
industry;  however,  we  focus  on  US  Air  Force  wargames  and  understand  the  role  that 
logistics  has  played  in  wargames.  This  section  illustrates  the  shortcomings  that  logistics 
has  had  in  wargames  and  the  numerous  issues  involved  with  these  shortcomings. 

Wargames  are  used  within  several  levels  of  the  Air  Force.  Each  level  provides 
different  types  of  insight,  but  we  concentrate  on  Title  10  wargames.  “As  the  Goldwater- 
Nichols  Act  of  1986  gave  the  service  chiefs  responsibility  under  U.S.  Code  Title  10  to 
train,  man,  and  equip  their  individual  forces”  (Ducharme,  2012: 1).  The  Air  Force  began 
conducting  Title  10  wargames  in  1995  by  creating  two  types  of  wargames:  Unified 
Engagement  (UE)  and  Future  Capabilities  Game  (Ducharme,  2012:2).  UE  wargames  are 
completed  on  even  years  with  Future  Capabilities  Games  being  held  on  odd  years.  These 
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two  types  of  wargames  have  entirely  different  objectives,  but  we  strictly  focus  on  Unified 
Engagements. 

The  goal  of  UE  is  to  “address  military  challenges  and  concept  exploration”  within 
the  Pacific  and  European  theaters  (Ducharme,  2012:2).  The  wargames  are  conducted  in 
theater  and  are  based  on  technologies  that  are  near  tenn  (approximately  12  years  out). 

UE  is  established  to  be  a  tool  that  an  operational  commander  can  use  to  develop  and  test 
campaign  level  strategies  in  their  respective  theater  (Duchanne,  2012:43).  There  are 
three  teams  or  forces  involved  with  the  wargames.  The  white  team  adjudicates;  the  Blue 
Force  represents  the  US  forces  while  the  Red  Force  represents  the  enemy  forces  (Caffrey, 
2008:40).  The  players  during  these  wargames  are  the  decision  makers  and  are  using 
history,  culture  and  doctrine  to  develop  strategies  and  crisis  action  plans  (Caffrey, 
2008:43).  Each  UE  is  different  and  the  rules  for  winning  the  wargame  change;  however, 
the  main  goal  is  not  necessarily  to  win  the  wargame  but  to  come  out  with  useful  insights 
on  operational  strategies  (Haffa,  2001).  Because  of  the  complexity  of  typical  UE 
scenarios,  constructive  computer  simulations  are  used  to  capture  daily  Red  and  Blue 
Force  moves  as  well  as  adjudication  of  battlefield  results. 

One  of  the  main  issues  of  UE  is  a  failure  to  evaluate  the  logistic  operations 
(Haffa,  2001).  Logistics  is  regarded  as  an  operational  constraint  and  a  failure  to 
incorporate  logistics  has  an  impact  on  campaign  planning  (LaPlante,  1996:97).  Blue 
Forces  overestimate  their  effectiveness  level  and  completely  disregard  the  logistics 
involved  in  supporting  their  campaign.  For  example,  the  kill  rates  included  in  the 
simulations  require  that  munitions  be  provided  for  each  of  the  weapon  systems;  however, 
the  amount  of  ammunition  available  is  considered  unlimited  and  use  is  not  tracked  (Haffa 
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2001).  This  in  turn  over  predicts  the  effectiveness  of  US  forces,  under  predicts  the  red 
forces,  underestimates  the  budget,  and  provides  less  insightful  results  to  a  large  number 
of  policies  and  doctrine  (Caffrey,  2008:37). 

Other  reasons  that  logistics  has  failed  to  be  captured  in  wargames  is  because 
logisticians/warfighters  were  excluded  from  the  wargames  (LaPlante,  1996:97)  and  the 
cost/complexity  of  incorporating  logistics  is  far  too  great  (Duchanne,  2012:4).  Over  the 
last  couple  of  decades  logisticians  and  warfighters  have  been  incorporated  into 
wargames;  however,  there  were  still  issues  in  capturing  logistics  in  simulation  models 
(LaPlante,  1996:97).  The  main  reason  for  this  matter  was  still  the  issues  with  complexity 
and  cost  of  further  developing  the  wargame  models.  But  what  senior  leaders  fail  to 
understand  is  that  funding  a  project  to  develop  and  maintain  logistics  within  a  wargame 
simulation  model  is  far  less  expensive  than  underestimating  the  budget  for  a  war  in  the 
Pacific. 

LCOM-ATK  Overview 

In  an  effort  to  reduce  cost,  complexity  and  understanding  of  incorporating 
logistics  into  a  wargame  model,  we  felt  that  using  a  preexisting  logistics  model  as  the 
baseline  would  significantly  reduce  these  three  factors.  We  needed  a  logistics  model  that 
ran  at  the  operational  level  and  contained  all  of  the  support  personnel  and  equipment 
required  to  capture  Agile  Combat  Support.  LCOM-ATK  accomplished  all  of  these 
requirements  and  had  been  used  throughout  the  logistics  community  for  decades. 

The  model  constructed  for  this  paper  was  developed  under  the  Logistics 
Composite  Model  (LCOM)  Analysis  Toolkit  (ATK)  version  4.0  which  is  a  product  of 
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AFLCMC/EZJS  with  support  from  Frontier  Technology  Incorporated  (FTI).  LCOM- 
ATK  is  a  composite  of  modules  written  in  SIMSCRIPT  II,  which  communicate  with  each 
other  to  function  as  a  unit  (Erdman,  2014: 13).  The  three  modules  are:  the  Input  Module 
that  preprocesses  the  data,  the  Main  Module  that  runs  the  simulation  and  the  Post 
Processor  Module  that  analyzes  post  simulation  data  (Erdman,  2014:27).  The  software  is 
used  to  simulate  studies  concerning  AF  base  level  functions  like  maintenance  and  supply. 
LCOM  relies  on  the  fact  that  through  the  simulation,  the  operators  can  assess  the 
availability  of  support  resources  and  operational  weapon  systems  (Erdman,  2014:1 1). 

One  of  the  great  things  about  LCOM  is  that  it  leaves  a  significant  amount  of 
control  to  the  user  over  the  level  of  detail  that  the  model  environment  possesses.  This 
was  vital  in  our  research  because  of  the  conflicting  aggregation  levels  of  wargames  and 
constructive  logistic  models  used  for  analysis.  This  was  the  key  in  choosing  a  logistics 
model  as  our  base  model  as  opposed  to  a  wargame  model.  The  LCOM  model  is  designed 
to  process  spares  and  other  like  subsystems  to  capture  the  support  system  during  an 
engagement  (AFLCMC/EZJS,  2013).  This  allows  its  users  to  tailor  the  model  and  the 
three  modules  to  capture  a  wide  range  of  different  types  of  logistics  processes  and 
resources  while  maintaining  a  manageable  level  of  understanding  and  complexity.  The 
general  flow  for  the  three  modules  is  shown  in  Figure  1 . 
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The  output  of  LCOM-ATK  is  consolidated  into  reports.  The  reports  or  statistics 
collected  are  broken  into  groups  that  have  a  varying  number  of  measures  within  them 
(AFLCMC/EZJS,  2013).  We  used  all  of  the  measures  described  in  each  of  the  following 
groups  in  our  analysis,  but  please  note  that  they  are  only  a  subset  of  metrics  available 
within  LCOM-ATK.  A  complete  listing  of  the  measures  used  for  our  logistic  simulation 
can  be  found  in  Appendix  A  of  this  paper.  In  addition,  please  note  that  throughout  this 
paper  measures,  metrics,  and  statistics  are  equivalent. 

The  first  set  of  reports  is  Group  A  or  also  known  as  Key  Mission  Statistics 
(Erdman,  2014:28).  This  group  contains  the  metrics  that  provide  a  top  level  assessment 
of  the  missions  conducted  during  the  engagement  (AFLCMC/EZJS,  2013).  Metrics  that 
deal  with  completing  sorties,  attrition,  and  supply  wait  times  are  found  in  Group  A.  A 
subset  of  the  statistics  gathered  in  this  group  is  shown  in  Table  1. 
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Tablet.  Group  A  (Walker  et  al.,  2010) 


Stat 

Description 

A4 

Number  of  sorties  requested 

A5 

Number  of  sorties  initiated 

A7 

Number  of  Attritions 

All 

Average  aircraft  pre- sortie  maintenance  (hours) 

A16 

Average  aircraft  mission  wait  status  (hours) 

A20 

Average  aircraft  post-sortie  time  (hours) 

A26 

Number  of  sorties  completed 

The  next  set  of  statistics  is  Group  B  which  encompasses  all  Key  Activity 
Statistics  (Erdman,  2014:28).  This  group  contains  the  metrics  that  provide  a  view  of  all 
the  activities  that  are  started  in  the  simulation  (AFLCMC/EZJS,  2013).  Activities 
encompass  all  the  movement  of  entities  within  the  simulation.  Some  examples  of 
activities  are:  receiving  engines,  requesting  ammunition,  or  time  taken  to  receive 
maintenance  parts.  A  subset  of  the  statistics  gathered  in  this  group  is  shown  in  Table  2. 


Table  2.  Group  B  (Walker  et  al.,  2010) 


Stat 

Description 

B1 

Number  of  activities  requested 

B2 

Number  of  activities  started 

B3 

Number  of  activities  cancelled 

B4 

Average  time  to  get  resources  (hours) 

B8 

Average  activity  length  (hours) 

B12 

Number  of  activities  completed 

B13 

Number  of  exogenous  activities  requested 

Group  C  contains  the  Key  Aircraft  statistics  (Erdman,  2014:28).  This  group 
contains  the  metrics  that  provide  a  view  of  all  the  activities  that  are  associated  with  the 
aircraft  within  the  simulation  (AFLCMC/EZJS,  2013).  Group  C  focuses  on  metrics  such 
as  number  of  aircraft  available,  unscheduled  maintenance  conducted  on  the  aircraft,  and 
number  of  sorties  completed.  A  subset  of  the  statistics  gathered  in  this  group  is  shown  in 
Table  3. 
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Table  3.  Group  C  (Walker  et  aL,  2010) 


Stat 

Description 

Cl 

Number  of  aircraft  authorized 

C2 

Number  of  aircraft  days  available 

C3 

Percent  sorties  (including  alert) 

C4 

Percent  unscheduled  maintenance 

C5 

Percent  scheduled  maintenance 

C6 

Percent  NMCS 

C7 

Percent  time  waiting  to  fly 

The  Key  Personnel  Statistics  are  gathered  in  Group  D  (Erdman,  2014:28).  This 
group  contains  the  metrics  that  provide  a  view  of  all  the  activities  that  are  associated  with 
the  personnel  within  the  simulation  (AFLCMC/EZJS,  2013).  These  statistics  consider 
the  entire  maintenance  population  and  focus  on  metrics  incorporating  the  manhours  used 
for  activities.  A  subset  of  the  statistics  gathered  in  this  group  is  shown  in  Table  4. 

Table  4.  Group  D  (Walker  et  al.,  2010) _ 


Stat 

Description 

D1 

Manhours  available 

D2 

Percent  utilization 

D3 

Manhours  used 

D4 

Percent  used  for  unscheduled  maintenance 

D5 

Percent  used  -  scheduled  maintenance 

D8 

On- equipment  manhours  used 

D9 

Off-equipment  manhours  used 

Group  E  covers  the  Key  Shop-Repair  Statistics  (Erdman,  2014:28).  This  group 
contains  the  metrics  that  provide  a  view  of  all  the  activities  that  are  associated  with  shop 
repair  within  the  simulation  (AFLCMC/EZJS,  2013).  The  metrics  in  Group  E  are  meant 
to  show  how  well  the  shop  is  doing  in  repairing  the  aircraft.  Metrics  like  the  number  of 
items  backlogged,  the  repair  cycle  time,  and  number  of  items  in  repair  all  provide  a 
snapshot  of  the  repair  shops.  A  subset  of  the  statistics  gathered  in  this  group  is  shown  in 
Table  5. 
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Table  5.  Group  E  (Walker  et  aL,  2010) 


Stat 

Description 

El 

Number  of  rep  gens  or  exog  demands 

E4 

Average  base  repair  cycle  (days) 

E10 

Number  of  items  in  repair  (EOP) 

Ell 

Number  of  items  backlogged  (EOP) 

The  next  set  of  statistics  is  Group  F  which  contains  the  Key  Supply  Statistics 
(Erdman,  2014:28).  This  group  contains  the  metrics  that  provide  a  view  of  all  the 
activities  that  are  associated  with  the  supply  within  the  simulation  (AFLCMC/EZJS, 
2013).  The  metrics  in  Group  F  are  meant  to  show  how  well  the  supply  side  of  logistics  is 
perfonning  using  metrics  such  as  the  number  of  backorder  days  for  a  part,  number  of 
units  demanded,  and  the  number  of  cannibalizations.  A  subset  of  the  statistics  gathered 
in  this  group  is  shown  in  Table  6. 

Table  6.  Group  F  (Walker  et  aL,  2010) _ 


Stat 

Description 

FI 

Authorized  quantity 

F3 

Number  of  backorder  days 

F4 

Number  of  units  demanded 

F8 

Percent  demands  not  satisfied 

F9 

Number  of  cannibalizations 

FI  1 

NMCS  indicator 

The  Key  Equipment  Statistics  are  captured  in  Group  G  (Erdman,  2014:28).  This 
group  contains  the  metrics  that  provide  a  view  of  all  the  activities  that  are  associated  with 
the  maintenance  equipment  within  the  simulation  (AFLCMC/EZJS,  2013).  The  metrics 
in  Group  G  are  meant  to  show  how  well  the  maintenance  equipment  is  holding  up  for  the 
simulation.  Are  their  issues  with  not  having  enough  maintenance  equipment  to  maintain 
the  sorties  being  conducted?  Questions  like  this  can  be  answered  investigating  the 
metrics  in  Group  G.  A  subset  of  the  statistics  gathered  in  this  group  is  shown  in  Table  7. 
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Table  7.  Group  G  (Walker  et  aL,  2010) 


Stat 

Description 

G1 

Authorized  quantity 

G2 

Equipment  hours  available 

G6 

Equipment  hours  used 

G9 

Number  of  backorder  days 

G10 

Number  of  units  demanded 

G16 

Percent  demands  not  satisfied 

G18 

Number  of  units  generated 

The  final  set  of  metrics  used  for  our  wargame  model  is  Group  H  which  consists  of 
the  Key  Facility  Statistics  (Erdman,  2014:28).  This  group  contains  the  metrics  that 
provide  a  view  of  all  the  activities  associated  with  the  maintenance  facilities  within  the 
simulation  (AFLCMC/EZJS,  2013).  The  metrics  in  Group  H  are  meant  to  indicate  the 
perfonnance  level  of  the  maintenance  facilities.  Are  there  enough  maintenance  facilities 
for  the  aircraft  to  be  repaired  or  are  there  too  many  maintenance  facilities  that  are  not 
being  utilized?  Questions  like  these  can  be  answered  with  the  metrics  found  in  Group  H. 
A  subset  of  the  statistics  gathered  in  this  group  is  shown  in  Table  8. 

Table  8.  Group  H  (Walker  et  aL,  2010) _ 


Stat 

Description 

H6 

Facility  hours  used 

H9 

Number  of  backorder  days 

H16 

Percent  demands  not  satisfied 

H17 

Average  hours  used/demand 

H19 

Facility  hours  backlog  (EOP) 

Logistics  Simulation 

Capturing  logistics  within  wargames  also  requires  a  decision  of  when  the  logistics 
simulation  should  be  conducted.  What  are  the  advantages  and  disadvantages  of  running 
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the  logistics  simulation  before,  during,  or  after  the  wargame?  In  order  to  answer  this 
question  we  first  investigate  what  the  goal  of  a  wargame  is. 

As  stated  previously,  wargames  are  a  tool  for  commanders  to  test  combat 
strategies  and  understand  the  capabilities  of  enemy  forces.  They  are  not  necessarily 
designed  to  perfonn  sensitivity  analysis  or  optimization.  The  loss  of  not  capturing 
logistics  within  wargames  primarily  influences  Blue  Force  capabilities.  There  are  various 
constraints  that  are  lost  when  not  considering  logistics.  For  example,  a  Blue  Force 
Commander  might  overcome  an  enemy  by  using  ten  F-35s;  but  in  actuality,  the  air  base 
that  the  F-35s  originate  from  can  only  maintain  and  operate  eight  F-35s  due  to  the 
amount  of  maintenance  facilities.  This  example  of  an  overestimation  of  Blue  Force 
capabilities  may  lead  to  budget  overruns,  failing  missions,  loss  of  airmen,  etc.  Thus  the 
role  of  logistics  is  to  further  define  a  near  actual  representation  of  Blue  Force  capabilities 
by  providing  limitations  and  constraints. 

Now  that  we  have  defined  the  role  of  logistics  within  wargames,  we  can 
detennine  when  would  be  the  most  appropriate  time  to  run  a  logistics  simulation.  The 
most  ideal  scenario  would  be  to  incorporate  a  logistics  simulation  to  inform  the  decisions 
that  the  Blue  and  Red  Force  commanders  make  during  the  wargame.  This  would  provide 
the  commanders  with  real  time  results  and  identify  strategies  that  are  feasible  or 
infeasible.  The  down  side  to  running  the  logistics  simulation  during  the  wargame  is  that 
the  setup  and  run  times  for  logistics  models  are  very  high.  Completing  a  logistics 
simulation  during  an  eight  hour  period  would  be  highly  unlikely.  Thus,  a  metamodel 
would  need  to  be  constructed.  This  metamodel  would  need  to  reduce  the  amount  of 
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variables,  and  in  turn,  provide  a  reasonable  run  time  that  would  allow  the  logistics  model 
to  be  completed  during  the  wargame. 

A  more  likely  scenario  would  be  to  run  the  logistics  simulation  before  or  after  the 
wargame.  In  both  of  these  cases,  commanders  are  provided  with  more  realistic 
capabilities  and  understand  the  limitations  of  their  respective  forces.  If  the  model  was  to 
be  executed  after  the  wargame,  then  the  logistics  simulation  could  be  modified  to 
incorporate  the  wargame  commanders’  strategies  implemented  during  the  wargame.  This 
would  allow  the  logistics  simulation  to  determine  the  feasibility  of  each  of  their 
strategies.  However,  these  feasibility  results  are  provided  after  the  wargame  is  concluded 
which  doesn’t  provide  commanders  with  the  opportunity  to  alter  their  approach. 

Because  of  this  reason,  we  feel  that  the  most  appropriate  time  to  capture  logistics 
within  wargames  is  before  the  wargame  is  played.  The  commanders  can  adjust  their 
strategies  during  the  wargame  because  they  are  provided  with  logistics  limitations  and 
constraints  before  the  wargame  begins.  This  is  a  very  important  benefit  because  this 
information  can  provide  better  insights  from  wargames.  As  with  any  case,  there  are 
always  downfalls  to  any  approach.  The  disadvantage  for  this  case  is  that  the  model 
would  not  have  the  ability  to  incorporate  the  decisions  made  during  the  wargame.  But  as 
we  stated  previously,  the  goal  of  logistics  being  incorporated  into  wargames  is  to  provide 
more  realistic  capabilities  of  the  forces.  Completing  the  logistics  simulation  before  the 
wargame  does  this  and  provides  the  wargame  commanders  with  information  at  an  ideal 
moment  in  time.  We  carry  this  approach  a  step  further  in  later  discussions,  developing  a 
metamodel  from  our  logistics  simulation  to  be  used  during  the  wargame. 
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The  next  step  after  selecting  when  our  logistics  simulation  would  be  conducted  is 
to  construct  the  fictional  scenario  our  wargame  would  portray.  The  problem  set  forth  for 
us  through  AFMC  was  to  assist  and  provide  insight  on  decisions  made  for  the  “Pivot  to 
the  Pacific.”  This  is  the  idea  that  the  US  would  shift  its  focus  from  the  Middle  East  to  the 
Pacific  region.  Due  to  security  concerns,  the  scenario  for  the  wargame  is  completely 
fictional.  There  are  no  real  opposing  forces,  the  flight  times  used  to  conduct  sortie 
operations  are  not  real,  and  the  locations  of  both  Blue  and  Red  Forces  do  not  reflect 
reality.  The  idea  of  our  study  is  to  provide  a  proof  of  concept  that  a  stand-alone  logistics 
simulation  can  effectively  capture  a  more  accurate  representation  of  the  impact  of  ACS 
for  a  standard  wargame  scenario  in  a  timely  manner  (before  or  during  wargame  play). 

The  scenario  takes  place  in  the  Pacific  with  three  Blue  Force  units  and  two  Red 
Force  units.  The  forces  are  captured  in  LCOM-ATK  by  having  Blue  forward  operating 
bases  (FOB)  each  conducting  sorties  via  F-35s  over  two  regions  controlled  by  the  Red 
Forces.  We  constructed  the  model  using  a  predefined  set  of  Blue  Force  strategies  to 
conduct  air  operations  over  a  180  day  period. 

The  lengths  of  UEs  are  typically  4-14  days,  but  we  selected  an  extended  period  of 
180  days  because  logistics  usually  cannot  be  captured  in  a  short  window  of  time.  For 
example,  running  a  scenario  that  lasted  seven  days  wouldn’t  capture  logistics  because  the 
logistics  operations  would  be  near  zero.  The  question  that  then  arises  is  at  what  point  is 
logistics  relevant?  Due  to  the  scope  and  timeframe  of  this  study,  we  feel  that  assuming 
180  days  would  be  sufficient,  but  further  analysis  could  determine  the  actual  minimum 
amount  of  time  needed  to  capture  logistics.  Using  this  extended  period  of  time  also 
allows  the  users  with  the  ability  to  extract  a  set  of  days  matching  the  wargame  length 
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from  any  point.  This  then  could  detennine  what  effects  logistics  would  play  during  the 
wargame  and  provide  more  accurate  starting  logistics  capabilities.  In  addition,  as  we 
have  seen  with  the  wars  with  Iraq  and  Afghanistan,  our  future  engagement  planning 
should  include  a  sustainment  piece.  Thus  selecting  a  time  period  of  180  days  is 
reasonable  because  it  allows  logistics  to  be  captured  during  both  a  surge  and  sustainment 
window. 

The  three  Blue  FOBs  each  contain  24  Joint  Strike  Fighters  (JSF),  3  maintenance 
facilities,  6  maintenance  crew  members,  ammunition,  fuel  and  a  vast  amount  of  parts  to 
repair  on  the  JSF.  LCOM-ATK  incorporates  thousands  of  parts  that  can  fail  on  the  JSF, 
but  we  want  to  focus  specifically  on  ACS  by  incorporating  engine  failures.  The  failure  of 
missions  due  to  unscheduled  maintenance  and  backorder  days  of  engine  spares  provides 
more  realistic  force  capabilities.  We  want  to  explore  questions  like  how  many  days 
would  a  JSF  be  unable  to  operate  or  what  is  the  impact  to  day  to  day  operations  if  an 
engine  failure  occurs.  Tracking  engine  failures  and  spares  allows  us  to  accomplish  this 
and  provides  a  sufficient  representation  of  ACS  within  a  wargame  scenario. 

The  Red  Forces  have  no  ability  to  defend  against  the  JSF  sortie  operations  but 
control  two  ground  regions  in  the  Pacific.  Due  to  this  aspect,  Red  Forces  are  not  able  to 
attrite  Blue  Forces,  however,  this  could  be  easily  modified  within  LCOM-ATK.  In 
addition,  we  felt  that  this  didn’t  impact  our  overall  goal  for  this  thesis,  which  was  to 
capture  logistics  in  a  wargame  model.  Figure  2  provides  a  visual  representation  of  the 
logistic  simulation  scenario. 
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Figure  2.  Wargame  Scenario 

A  sortie  completed  by  the  Blue  Force  is  considered  successful  if  the  JSF  is  able  to 
load  ammo,  refuel,  pass  maintenance  inspections,  takeoff,  reach  the  Red  base,  and  return 
to  the  Blue  base  it  originated  from.  The  sortie  operations  vary  in  length  based  on  the 
origin  of  the  JSF.  Figure  2  shows  the  distances  from  each  of  the  bases,  while  Figure  3 
captures  the  sortie  operations  modeled  within  LCOM-ATK.  We  are  not  concerned  with 
the  ammunition  hitting  the  desired  mark  or  destroying  a  target,  but  are  concerned  with  the 
amount  of  ammunition  available.  Specifically,  we  focus  on  if  enough  ammunition  is 
available  to  complete  the  sortie.  We  are  not  varying  the  types  of  ammunition,  but 
LCOM-ATK  can  be  altered  to  incorporate  various  types  of  ammunition.  The  goal  of  the 
logistic  simulation  for  each  sortie  operation  is  to  focus  on  the  logistic  support  required  to 
complete  that  specific  mission. 
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Figure  3.  Simulation  Model  (Erdman,  2014:5) 


The  logistics  simulation  has  time  steps  of  24  hours  and  a  time  length  of  180  days. 
The  time  step  is  selected  for  1-day  intervals  because  that  is  customarily  the  length  for 
logistic  models  and  how  real  world  metrics  are  gathered.  We  selected  10  replications 
because  it  provides  a  reasonable  95%  half  width  for  statistics  collected. 


Analysis  Methodology 

Logistics  models  typically  are  run  at  a  low  aggregation  level  and  have  high 
fidelity.  What  this  corresponds  to  is  an  output  of  a  large  number  of  variables  and  data 
points.  For  example,  our  wargame  scenario  contained  three  Blue  Force  units  and  two 
Red  Forces  units  with  logistic  support  for  the  Blue  Forces.  The  Blue  Force  had  one 
squadron  of  JSFs  (24  x  F-35)  located  at  each  FOB  with  ACS.  This  resulted  in  an  output 
of  well  over  100  metrics  with  100,000  data  points.  Presenting  this  to  a  group  of  decision 
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makers  within  a  wargame  is  a  daunting  task  because  you  want  to  infonn  decisions 
without  overwhelming  them. 

We  are  capturing  logistics  in  wargames  to  provide  more  realistic  force 
capabilities.  Our  analysis  focuses  on  providing  trends  and  limitations  of  the  logistics 
simulation  (e.g.  sortie  rates,  engine  failures,  F-35  downtimes,  scheduled  maintenance, 
available  ammunition,  etc.).  We  showcase  what  ACS  can  provide  during  surge  and 
sustainment  phases  of  operations.  These  results  are  then  provided  to  decision  makers 
within  wargames  to  understand  what  their  force  limitations  are. 

In  addition,  we  explore  the  feasibility  of  developing  a  metamodel  that  can  be 
completed  during  a  wargame.  We  use  various  Multivariate  Analysis  techniques 
(Principal  Component  Analysis,  Discriminant  Analysis,  Factor  Analysis,  and  Artificial 
Neural  Networks)  to  reduce  the  dimensionality  while  maintaining  minimal  loss  of 
information.  The  benefits  of  having  the  ability  to  run  logistics  metamodels  during 
wargames  are  immense.  Commanders  are  given  near  real-time  feedback  on  their 
decisions  and  can  alter  their  strategies  throughout  the  wargame. 

Conclusion 

The  size  of  logistic  models  and  the  time  constraints  imposed  during  wargame 
play,  require  investigating  whether  there  are  alternative  approaches  to  running  a  logistics 
simulation  during  a  wargame  to  capture  ACS  within  a  wargame.  Specifically,  should  the 
logistics  model  be  run  before,  during,  or  after  the  wargame?  Completing  the  model 
during  the  wargame  is  the  ideal  approach  and  is  plausible,  but  is  highly  unlikely  because 
of  the  time  constraints  and  modifications  to  the  model.  A  metamodel  would  need  to  be 
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developed  that  can  capture  ACS  with  a  small  subset  of  logistic  inputs  and  allow  the 
model  to  run  significantly  faster.  The  most  practical  approach  is  to  complete  the  logistics 
simulation  before  the  wargame.  Completing  the  model  at  this  time  allows  logistics 
considerations  to  be  included  to  provide  commanders  with  more  realistic  force 
capabilities  and  to  gain  better  insights  from  wargames.  The  next  chapter  of  this  thesis 
analyzes  these  two  approaches  by  using  statistical  and  multivariate  analysis. 
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IV.  Analysis  &  Results 


The  analysis  section  for  this  study  is  broken  into  two  sub-categories.  The  first  sub¬ 
category  focuses  on  gathering  the  output  data  from  the  logistics  simulation  to  develop 
further  insights  for  the  wargame.  The  analysis  utilizes  graphs,  95%  confidence  intervals, 
and  hypothesis  testing  to  better  define  the  effectiveness  and  limitations  of  Blue  Forces. 
The  second  sub-category  explores  the  development  of  a  metamodel  to  provide  a  suitable 
logistics  model  to  be  run  during  the  wargame.  This  part  of  our  analysis  uses  Principal 
Component  Analysis  (PCA),  Discriminant  Analysis  (DA),  Factor  Analysis  (FA)  and 
Artificial  Neural  Networks  (ANN)  on  the  output  data  from  the  logistics  simulation  to 
construct  a  metamodel. 

The  output  data  from  the  logistic  simulation  is  in  the  fonn  of  hundreds  of  thousands 
of  points  with  140  MOEs.  We  partition  the  data  into  three  groups  to  allow  for  testing  and 
validation.  We  use  70%  of  the  data  to  perfonn  analysis,  15%  for  testing,  and  15%  for 
validation.  In  addition,  the  output  data  is  not  balanced  in  terms  of  the  response  variable 
(sortie  completion  or  not).  We  achieve  approximately  85%  success  and  15%  failure 
rates.  An  unbalanced  set  of  data  affects  both  sub-categories  of  analysis,  but  we  explicitly 
explain  the  issues  that  arise  within  the  multivariate  analysis  discussion. 

The  training  and  testing  of  the  data  is  captured  using  the  hold-out  method  for  cross- 
validation  (Devijver  and  Kittler,  1982:10).  This  method  breaks  the  data  into  three  groups 
to  train  the  predictor,  test  the  predictor,  and  validate  the  predictor.  As  stated  previously, 
we  divided  the  output  data  into  7 0%/ 1 5  %/ 15%  for  training,  testing,  and  validation.  More 
specifics  on  how  we  partitioned  the  data  are  discussed  with  our  multivariate  analysis. 
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We  use  all  of  these  different  types  of  analysis  to  demonstrate  that  incorporating 
logistics  into  wargames  provide  commanders  with  more  effective  insights.  These 
insights  carry  over  into  many  facets  within  the  DoD  that  impact  planning,  development, 
and  procurement  of  future  weapon  systems. 

Statistical  Analysis 

A  summary  of  the  sorties  completed  during  a  single  replication  of  the  logistics 
simulation  is  provided  in  Figure  4.  This  plot  showcases  the  sortie  completion  rate  for 
each  Blue  base  throughout  the  180  day  campaign.  We  chose  to  display  a  single 
replication  because  it  shows  significant  differences  in  sortie  completion  rates  by  base. 
An  average  of  the  ten  replications  does  not  display  this  characteristic  because  of  the 
randomness  involved  with  LCOM-ATK  and  the  fact  that  each  replication  is  independent 
of  each  other.  We  arbitrarily  chose  this  replication  to  provide  insight,  and  it  is  not  more 
significant  than  any  of  the  other  replications. 

Sortie  Completion  Rate 
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Figure  4.  Sortie  Completion  Rate 
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The  graph  in  Figure  4  shows  that  all  three  bases  start  off  relatively  high  in  their 
respective  sortie  completion  percentages  (90-100%),  but  begin  to  draw  down  throughout 
the  campaign.  Each  of  the  Blue  bases  shows  that  they  are  completing  approximately 
85%  of  their  missions  after  the  95th  day.  The  average  of  the  ten  replications  for  sortie 
completions  agreed  with  this  finding  (85%  mission  success  rate),  and  achieved  an 
average  success  rate  of  83%.  Another  insight  from  the  single  replication  plotted  in 
Figure  4  is  that  there  are  spikes  indicating  significant  drops  in  completing  missions. 
These  drops  in  the  plot  can  come  from  any  number  of  things,  but  a  good  initial  area  to 
explore  is  in  unscheduled  maintenance. 

A  spike  might  occur  because  of  unscheduled  maintenance  due  to  engine  failures 
or  other  parts  needing  repair  on  the  JSF.  These  unscheduled  events  do  not  allow  the  JSF 
to  be  operationally  ready,  and  in  turn,  not  complete  the  mission.  Figure  5  showcases  the 
JSFs  unscheduled  maintenance  rates  for  each  Blue  base  throughout  the  180  day 
campaign.  Please  note  that  Figure  5  is  the  unscheduled  maintenance  rate  for  the  same 
single  replication  used  in  Figure  4. 
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Figure  5.  JSF  Unscheduled  Maintenance  Rate 
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Figure  5  does  have  spikes  that  indicate  a  significant  increase  in  unscheduled 
maintenance  on  the  same  days  as  the  drops  in  sortie  completion  rates  as  shown  in  Figure 
4.  For  example,  on  day  3 1  Base  B  completed  approximately  72%  of  its  missions  that 
day.  This  was  a  drop  of  28%  in  one  day.  Day  3 1  on  Figure  5  shows  a  large  increase  in 
unscheduled  maintenance  for  that  day.  Insights  like  this  can  be  found  looking  throughout 
plots,  but  it  is  time  consuming  and  difficult  to  provide  significance  and  understanding  of 
why  things  are  occurring. 

We  transfer  over  to  conducting  statistical  analysis  on  the  logistics  simulation 
output  data  and  determine  if  there  are  added  insights  that  can  be  provided  to  wargame 
commanders.  Some  key  statistics  are  provided  in  Table  9.  These  statistics  are  gathered 
from  the  entire  set  of  replications  from  the  logistics  simulation. 


Table  9.  Logistics  Simulation  Analysis 


Base  A 

Base  B 

Base  C 

Mean 

St  Dev 

95%HW 

Mean 

St  Dev 

95%HW 

Mean 

St  Dev 

95%HW 

Sortie  Comp  (%) 

87.5 

4.6546 

2.8849 

88 

5.2813 

3.2733 

91.9 

7.0658 

4.3793 

Sortie  Comp  (#) 

7.4722 

3.4405 

2.1324 

7.4836 

3.3188 

3.5357 

7.8277 

3.5357 

2.1914 

JSF  Op  Ready  (%) 

35.446 

10.463 

6.4848 

37.422 

10.419 

6.4577 

39.589 

10.382 

6.4346 

Unsched  Mx  (%) 

8.4 

4.5018 

2.7902 

7.6 

4.1539 

2.5746 

6.5 

2.8659 

1.7763 

Sched  Mx  (%) 

12.682 

3.3517 

2.0773 

11.276 

3.5058 

2.1729 

9.8136 

3.4598 

2.1444 

Msn  Wait  Time  (hr) 

0.5675 

0.0552 

0.0342 

0.5179 

0.0665 

0.0412 

0.4884 

0.0684 

0.0424 

Table  9  showcases  the  mean,  standard  deviation  and  95%  half  width  for  each  of 
the  Blue  bases.  There  is  a  trend  with  each  of  the  MOEs  displaying  Base  C  performing 
better  than  Base  B  and  Base  A.  This  might  occur  because  of  the  distances  involved  with 
each  of  the  Blue  bases  and  their  Red  ground  area  to  attack.  The  greatest  change  between 
the  Blue  bases  in  Table  9  is  the  sortie  completion  rate.  Base  A  was  completing 
approximately  87.5%  sorties  while  Base  C  was  completing  91.9%  sorties.  This  is 
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interesting  because  you  would  assume  that  all  the  bases  would  perfonn  at  around  the 
same  level,  but  the  distance  of  sorties  might  be  a  factor  in  this  case  as  well. 

We  conduct  a  hypothesis  test  in  which  our  null  hypothesis  is  that  Base  A  has  the 
same  sortie  completion  rate  as  Base  C.  We  use  a  two  sample  paired  T-test  at  an  alpha 
level  of  .05  with  no  assumptions  required  for  the  two  samples  variances.  After 
perfonning  the  test,  we  reject  the  null  hypothesis  because  our  T  statistic  is  -6.996,  which 
is  less  than  our  T  critical  value  of -1.968.  This  implies  that  the  sortie  completion  rate  for 
Base  A  is  not  equal  to  sortie  completion  rate  of  Base  C;  however,  the  difference  in  means 
of  the  two  bases  is  low  (4.4%).  We  investigate  further  by  producing  a  95%  confidence 
interval  (Cl)  on  the  difference  of  means  of  sortie  completion  rates  for  both  Base  A  and  C. 
We  obtained  a  95%  Cl  of  [3.1597,  5.6403]  which  indicates  that  the  means  are  in  fact  not 
equal  because  the  interval  does  not  contain  zero,  and  that  Base  C  is  better  than  Base  A  by 
about  5%  in  sortie  completion  rates.  This  insight  might  drive  an  analysis  to  be  conducted 
on  why  is  there  a  difference  between  the  bases  and/or  detennine  whether  a  FOB  should 
even  be  located  in  Base  A. 

We  then  conduct  a  hypothesis  test  in  which  our  null  hypothesis  is  that  Base  A  has 
the  same  JSF  operational  rate  as  Base  C.  This  metric  detennines  how  many  JSF  are 
operationally  ready  and  can  complete  sortie  operations  on  a  daily  basis.  We  use  a  two 
sample  paired  T-test  at  an  alpha  level  of  .05  with  no  assumption  required  for  the  two 
samples  variances.  After  perfonning  the  test,  we  reject  the  null  hypothesis  because  our  T 
statistic  is  -3.77  and  is  less  than  our  T  critical  value  of -1.968.  This  implies  that  the  JSF 
operational  rate  for  Base  A  is  not  equal  to  JSF  operational  rate  of  Base  C.  Constructing  a 
95%  confidence  interval  on  the  null  hypothesis  that  the  difference  in  means  of  JSF 
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operational  ready  rates  is  zero  provides  more  insight.  We  obtained  a  95%  Cl  of  [3.461, 
4.8234]  which  indicates  that  the  means  are  in  fact  not  equal  because  the  interval  does  not 
contain  zero,  and  that  Base  C  is  better  than  Base  A  in  JSF  operationally  ready  rate.  This 
insight  might  alter  a  wargame  commander’s  strategy  in  not  utilizing  Base  A  as  much  and 
focus  on  conducting  more  operations  with  Base  C. 

This  small  subset  of  statistical  testing  demonstrates  the  type  of  information  we 
can  provide  commanders  during  wargames  to  more  accurately  represent  their  forces  and 
capabilities.  For  example,  commanders  might  be  conducting  operations  with  three  full 
JSF  squadrons  per  day.  This  would  result  in  72  sortie  operations  in  a  day;  however,  if 
you  incorporate  ACS  into  the  wargame,  our  logistics  simulation  showcased  that  only 
approximately  24  sorties  can  be  accomplished  per  day.  This  is  a  very  important  insight 
because  the  number  of  JSF  squadrons,  ammunition,  available  engine  spares,  maintenance 
crew  members,  amount  of  fuel,  etc.  would  need  to  be  increased  to  maintain  the  Blue 
Force  capabilities  captured  in  the  wargame.  In  either  case,  the  wargame  commander  may 
alter  their  strategy  with  this  more  accurate  infonnation  gathered  from  incorporating  ACS 
capabilities. 

Multivariate  Analysis 

The  purpose  of  this  analysis  is  to  develop  a  data  screening  approach  that  results  in 
validated  predictive  metamodels.  We  use  various  multivariate  analysis  techniques  to 
provide  decision  quality  infonnation  from  the  output  data  to  assist  the  decision  makers  in 
making  informed  judgments  in  an  expeditious  timeframe.  The  goal  of  each  of  the 
techniques  we  discuss  is  to  better  understand  the  observations  (output  variables)  from  our 
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simulation  and  develop  approaches  to  more  efficiently  and  effectively  use  the 
observations  in  evaluating  selected  MOEs.  We  select  sortie  completion  as  the  response 
variable  for  our  study;  however,  many  other  MOEs  are  available  within  LCOM-ATK  for 
a  similar  analysis. 

The  logistic  simulation  output  database  consists  of  140  variables  that  captured 
ACS  over  180  days  in  the  Pacific  region.  The  data  gathered  is  completely  fictitious,  but 
the  scenario  modeled  in  LCOM-ATK  could  be  quickly  and  easily  altered  to  represent  an 
actual  wargame  scenario. 

We  partition  the  data  into  three  groups  to  allow  for  testing  and  validation.  We  use 
70%  of  the  data  to  perform  analysis,  15%  for  testing,  and  15%  for  validation.  Each  data 
point  contains  output  for  a  single  replication  over  180  days.  The  data  points  are 
randomly  selected  using  the  Excel  random  function  by  assigning  each  data  point  a 
random  number  between  0  and  1.  The  top  70%  are  used  for  analysis,  the  next  15%  for 
testing,  and  the  bottom  15%  for  validation.  We  then  ensure  that  each  set  of  data  contains 
approximately  equal  balances  in  tenns  of  success  and  failures  of  the  response  variable 
(sortie  completion  or  not).  We  achieve  approximately  85%  success  and  15%  failure  rates 
for  each  of  the  sets.  An  unbalanced  set  of  data  may  affect  the  multivariate  analysis 
techniques  in  correctly  predicting  sortie  completed  (SC)  or  not  (SNC).  The  models  may 
tend  to  gravitate  towards  the  heavier  weighted  response  and  continue  to  predict  that 
response,  which  in  turn,  may  skew  the  prediction  levels.  We  address  this  issue  within 
each  of  the  multivariate  analysis  techniques. 
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Principal  Component  Analysis  (PCA) 

The  first  technique  performed  is  Principal  Component  Analysis  (PCA)  which  is  a 
statistical  method  that  determines  which  principal  components  are  important  to  the 
observations  or  simulation  output  variables.  Principal  components  (PC)  are  orthogonal 
transformations  of  given  correlated  variables  that  transfonn  into  a  set  of  linearly 
uncorrelated  variables  (Jolliffe,  2002: 1 1).  In  other  words,  PCA  uses  eigenvectors 
(direction  of  the  data)  and  eigenvalues  (variance  of  the  data)  to  detennine  which  are  the 
highest  scores.  The  eigenvector  with  the  highest  eigenvalue  becomes  a  principal 
component  (Abdi  and  Williams,  2010:436).  Essentially,  PCs  are  the  directions  of  the 
data  where  there  is  the  most  amount  of  variance.  The  purpose  of  PCA  is  to  reduce  the 
dimension  of  the  observations  by  identifying  patterns  with  minimal  loss  of  information 
(Jolliffe,  2002:9). 

PCA  was  conducted  on  the  output  data  with  no  issues  encountered  while 
perfonning  this  analysis.  All  variables  and  data  points  were  included  in  the  analysis. 

The  first  step  in  perfonning  PCA  is  determining  how  many  components  should  be 
included  in  the  metamodel.  Horn’s  test  provides  this  answer  and  is  shown  in  Figure  6. 
All  Principal  Component  (PC)  scores  above  the  blue  line  pass  the  test  and  are 
recommended  for  the  metamodel. 
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Horn’s  test  shows  that  27  PCs  passed  and  thus  we  continue  the  analysis  with  this 
subset  of  PCs.  In  addition  to  this  test,  Kaiser’s  criterion  was  calculated  and  received  a 
score  of  0.9688  which  is  categorized  as  “Marvelous.”  To  ensure  a  minimal  loss  of 
information,  an  analysis  on  the  variance  explained  by  the  PCs  is  captured  in  Table  10. 
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Table  10.  PC  A  Variance  Explained 


Variance  Explained  by  PC 

Porportion  Cumalative 

PCI 

0.18251 

0.18251 

PC2 

0.08126 

0.26377 

PC3 

0.07954 

0.34331 

PC4 

0.07747 

0.42078 

PCS 

0.06678 

0.48756 

PC6 

0.05654 

0.5441 

PC7 

0.04642 

0.59052 

PC8 

0.04431 

0.63483 

PC9 

0.03394 

0.66877 

PC10 

0.03306 

0.70183 

PC11 

0.03178 

0.73361 

PC12 

0.02546 

0.75907 

PC13 

0.01376 

0.77283 

PC14 

0.01102 

0.78385 

PC15 

0.01076 

0.79461 

PC16 

0.01002 

0.80463 

PC17 

0.00987 

0.8145 

PC18 

0.00974 

0.82424 

PC19 

0.00952 

0.83376 

PC20 

0.00874 

0.8425 

As  shown  in  Table  10,  the  first  16  PCs  explain  approximately  80%  of  the 
variability,  which  is  an  acceptable  level.  We  then  compute  the  loadings  matrix  with  16 
principal  components  and  determine  which  scores  are  the  highest  values  in  each  of  the 
respective  rows.  Interpreting  these  results  provides  no  insight  on  patterns;  however,  we 
have  significantly  reduced  the  dimension  size.  We  now  move  into  detennining  if  patterns 
are  fonned  between  SC  and  SNC  through  the  various  principal  components. 

The  final  step  in  PCA  is  to  plot  the  PC  loading  scores  amongst  each  other  and 
detennine  if  there  is  a  set  of  PCs  that  developed  patterns  to  distinguish  between  SC  and 
SNC.  Figure  7  shows  the  best  combination  of  PCs  with  PCI  vs.  PC9;  however  there  are 
no  clear  patterns  that  distinguish  between  the  two  classes.  These  plots  can  distinguish 
sorties  not  completed  from  sorties  completed  but  not  the  reverse. 
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Figure  7.  PCI  vs.  PC9 


The  validation  set  produced  the  plot  in  Figure  8.  This  plot  represents  the  same 
process  and  methodolgy  that  the  training  set  underwent.  The  goal  of  the  validation  plot  is 
to  determine  whether  the  patterns  formed  in  the  training  set  mimic  the  patterns  fonned  in 
the  validation  plot. 
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Figure  8.  PCI  vs.  PC9  Validation 

As  Figure  8  showcases,  the  plot  matches  the  patterns  formed  in  Figure  7.  The 
plot  in  Figure  8  confirms  that  the  training  set  analysis  shown  in  Figure  7  is  correct  and 
validates  the  analysis. 

In  conclusion,  PCA  is  able  to  reduce  the  dimensionality;  however,  it  doesn’t  have 
the  ability  to  distinguish  between  the  two  classes  which  results  in  a  low  overall  prediction 
level.  In  addition,  an  interpretation  of  the  principal  components  is  not  conclusive  because 
of  the  immense  amount  of  variables  associated  with  each  of  the  PCs. 

Discriminant  Analysis  (DA) 

The  next  analysis  conducted  is  Discriminant  Analysis  (DA).  DA  is  a  statistical 
method  to  determine  a  categorical  dependent  variable,  such  as  sortie  completion,  by  one 
or  more  predictor  variables  (Johnson,  1992:39).  The  method  begins  with  gathering  a  set 
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of  observations,  in  which,  the  values  of  the  interval  variables  are  known.  This  produces  a 
training  set  which  is  then  used  to  determine  whether  the  developed  predicted  model  is 
correct  (Hardle,  2007:63).  The  goal  of  DA  is  to  decide  which  continuous  variables 
discriminate  between  two  or  more  groups  (Johnson,  1992:45).  This  approach  can  reduce 
dimensionality  and  still  maintain  a  high  level  of  prediction  in  distinguishing  if  a  sortie  is 
completed  or  not. 

In  performing  DA,  we  decided  to  remove  all  the  non-continuous  variables  due  to 
singularities  invalidating  matrix  multiplication.  We  remove  32  variables  from  the 
analysis  and  continue  with  108.  Conducting  the  analysis  produces  the  plot  captured  in 
Figure  9.  This  chart  demonstrates  the  amount  of  variables  removed  and  how  well  the 
model  is  predicting  the  result. 


Model  Efficiency 

0.9 


1  10  19  28  37  46  55  64  73  82  91  100 

Variables  Removed 

Figure  9.  DA  Model  Efficiency 

Figure  9  shows  that  with  67  variables  removed  (41  remaining),  the  model  is  able 
to  predict  approximately  61%  for  SC,  80%  for  SNC,  and  an  overall  prediction  level  of 
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70%.  We  believe  that  an  overall  prediction  level  of  75%  is  acceptable  which  occurred 
when  12  variables  removed  (96  remained).  This  analysis  shows  that  it  could  reduce  the 
dimensionality  and  continue  to  have  a  high  prediction  level;  however,  the  reduction  in 
dimensionality  wasn’t  as  far  as  we  would  have  liked.  We  are  hoping  to  develop  a 
metamodel  that  contains  less  than  5  variables  and  maintains  an  accurate  prediction  level 
of  no  less  than  75%. 

Factor  Analysis  (FA) 

The  third  technique  used  is  Factor  Analysis  (FA).  FA  is  a  statistical  method  to 
investigate  variable  commonality  and  factor  reduction  (Kim,  1978:  3).  FA  follows  the 
same  principals  as  PCA,  but  uses  a  rotated  loadings  matrix.  The  purpose  of  FA  is  to 
reduce  the  number  of  factors  and  to  detect  relationships  between  factors  by  classifying 
variables  (Thompson,  2004:  11).  It  also  provides  insight  in  grouping  factors  together  to 
understand  the  relationship  each  of  the  factors  have  in  the  overall  picture  of  the  model 
(Kim,  1978:  5). 

As  with  PCA,  the  same  process  is  in  place  and  produces  the  same  results  up  until 
the  loadings  matrix.  The  loadings  matrix  for  FA  contains  rotated  data  and  generates 
alternate  score  referred  to  as  Factor  scores.  As  shown  in  Table  11,  the  Factor  scores  are 
listed  in  the  loadings  matrix  and  the  highlighted  values  are  the  Factor  scores  with  the 
maximum  values  in  their  respective  rows.  Please  note  that  Table  1 1  contains  a  subset  of 
the  loadings  matrix.  The  full  loadings  matrix  is  too  large  to  include,  but  the  concept 
remains  the  same  for  the  entire  matrix. 
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Table  11.  FA  Loadings  Matrix 


Factor  Analysis  Loadins  Matrix 

Variable  Name 

Factor  1 

Factor  2 

Factor  3 

Factor  4 

Factor  5 

Factor  6 

Factor  7 

Factor  8 

Factor  9 

Factor  10 

NUMBER  OF  MISSIONS  REQUESTED 

-0.87376 

-0.0322 

-0.20068 

0.128385 

0.092952 

0.137436 

-0.11833 

-0.07903 

0.074627 

-0.14974 

NUMBER  OF  MISSION  ACCOMPLISHD 

-0.01798 

0.000921 

0.001566 

0.002161 

-0.04955 

-0.00854 

-0.04046 

0.048443 

0.024553 

0.906681 

PERCENT  ACCOMPLISHED 

-0.01534 

0.050909 

0.070919 

-0.07408 

0.846604 

0.049545 

0.024848 

0.052299 

0.026873 

0.104196 

NUMBER  OF  SORTIES  REQUESTED 

-0.15806 

-0.02693 

-0.17084 

0.84222 

-0.01777 

-0.12431 

-0.00114 

0.04269 

0.258558 

-0.03853 

NUMBER  OF  SORTIES  INITIATED 

-0.08824 

0.057075 

-0.06741 

0.046919 

-0.0124 

-0.89161 

-0.00345 

0.004176 

0.129233 

0.049135 

PERCENT  INITIATED 

-0.25941 

0.052607 

0.050216 

-0.01245 

-0.03472 

0.085072 

-0.09445 

0.054163 

-0.04711 

-0.84096 

NUMBER  OF  ATTRITIONS 

-0.37255 

0.069249 

-0.16634 

0.157325 

0.267184 

0.143814 

-0.55199 

0.192156 

-0.05367 

0.428328 

NUMBER  OF  RAM  REPAIRS 

-0.11119 

0.043984 

-0.85169 

0.122003 

-0.04783 

0.006695 

-0.04334 

0.180001 

-0.14612 

0.097564 

#  OF  AIR  ABORTS 

-0.00639 

-0.06047 

-0.15824 

0.154436 

-0.04361 

0.073285 

0.169765 

0.666974 

0.053432 

0.107297 

#  OF  SYMPATHETIC  AIR  ABORTS 

-0.17317 

-0.04281 

-0.03287 

0.045272 

0.849451 

0.016663 

-0.11541 

-0.09622 

-0.05693 

-0.04922 

AVG.  AC  PRESORTIE  MAINT.(HRS) 

0.152485 

-0.72978 

0.048473 

0.43812 

-0.04979 

0.248881 

-0.05276 

0.155454 

0.203792 

0.019288 

MIN  PRESORTIE  MAI  NT.  (HRS) 

0.053753 

-0.01358 

-0.83348 

0.185362 

-0.0285 

0.01394 

0.000864 

0.279428 

-0.17135 

0.01907 

MAX  PRESORTIE  MAINT.(HRS) 

-0.39165 

-0.07531 

-0.08866 

0.098373 

0.816687 

-0.09056 

-0.25054 

-0.08336 

0.01224 

-0.02302 

STD  DEV  PRESORTIE  MAINT  (HRS) 

-0.10007 

0.070705 

0.052926 

0.091354 

-0.04066 

-0.04007 

0.005346 

-0.08224 

0.65059 

0.032405 

NO.  OF  PRE  SORTIES  COMPLETED 

-0.24826 

-0.775 

0.042625 

-0.21462 

0.208495 

-0.00939 

-0.3549 

-0.00634 

0.048595 

-0.06792 

AVG.  AC  MISN  WAIT  STATUS(HRS) 

-0.89465 

-0.09904 

-0.2003 

0.166983 

0.108403 

-0.1537 

-0.08597 

-0.03374 

0.070513 

-0.08038 

MIN  MISN  WAIT  STATUS(HRS) 

-0.48507 

0.065273 

0.089642 

0.638633 

-0.00174 

-0.20097 

0.155321 

0.078754 

0.291334 

0.002626 

MAX  MISN  WAIT  STATUS(HRS) 

-0.0112 

-0.79123 

0.014659 

0.281998 

0.115033 

0.231398 

-0.30489 

0.045106 

0.190045 

-0.01986 

STD  DEV  MISN  WAIT  STATUS(HRS) 

0.225615 

-0.71571 

-0.06047 

0.01564 

-0.25811 

-0.21854 

0.402526 

0.190384 

-0.11511 

-0.06912 

The  next  step  is  interpreting  the  Factors  based  on  the  highlighted  values  in  each  of 
the  corresponding  Factors.  This  is  not  helpful  due  to  the  immense  size  of  the  loadings 
matrix  and  was  not  able  to  produce  an  insightful  interpretation  of  each  of  the  Factors. 

The  final  step  in  FA  is  to  produce  plots  that  compare  Factors  amongst  each  other  to 
determine  if  there  are  patterns  formed  to  distinguish  between  SC  and  SNC.  Such  a  plot  is 
shown  in  Figure  10. 
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Figure  10.  Factor  3  vs.  Factor  10 
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Figure  10  shows  that  FA  is  able  to  distinguish  SNC  from  SC  because  of  the  tight 
grouping  of  red;  however,  it  is  not  able  to  distinguish  SC  from  SNC  because  the  red 
points  are  intertwined  with  the  cluster  of  blue  points. 


Figure  11.  Factor  3  vs.  Factor  10  Validation 

Figure  1 1  is  the  validation  plot  for  FA.  This  plot  shows  that  it  mimics  the  same 
pattern  as  shown  in  Figure  10  and  thus  FA  is  validated.  However,  none  of  the  plots 
(including  Figure  10)  show  any  significant  distinct  patterns  being  formed.  There  is  no 
clear  distinction  between  red  and  blue  clusters  meaning  that  the  model  doesn’t  have  the 
ability  to  distinguish  between  SC  and  SNC.  We  conclude  that  FA  is  able  to  reduce  the 
dimensionality  of  the  variables,  but  it  isn’t  able  to  provide  an  accurate  prediction  level. 
Artificial  Neural  Network  (ANN) 

The  last  analysis  we  perfonn  is  Artificial  Neural  Networks  (ANN).  ANN  is  a 
statistical  model  that  estimates  functions  and  perfonns  pattern  recognition  (Schalkoff, 
1997:  2).  The  model  works  such  that  the  inputs  are  multiplied  by  weights  and  then 
computes  the  output  of  the  ANN  (Zurada,  1992:  36).  In  other  words,  there  are  nodes  and 
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interaction  of  nodes  referred  to  as  connections  in  ANNs.  The  nodes  are  simply  networks 
(or  variables)  that  receive  inputs  and  process  them  to  yield  outputs  (Hagan,  1996:21). 

The  connections  of  these  nodes  develop  a  global  behavior  that  we  cannot  see  because  of 
the  complexity  and  the  amount  of  nodes  (variables)  (Zurada,  1992:  37).  The  goal  of 
ANN  is  to  reduce  the  number  of  variables  while  maintaining  minimal  loss  of  information 
from  the  given  observation  set  (Yegnanarayana,  2009:3). 

ANN  is  a  methodology  that  continually  removes  variables  from  the  model  and 
outputs  its  prediction  levels  for  each  class  (SC  and  SNC).  We  utilize  ANN  by 
eliminating  the  variables  within  a  model  and  output  its  classification  accuracy  in  a  Signal 
to  Noise  Ratio  (SNR)  plot.  This  plot  is  found  in  Figure  12. 


Signal  to  Noise  Ratio 

100 


Figure  12.  ANN  Signal  to  Noise  Ratio 

The  full  model  is  able  to  predict  at  the  87.3%  prediction  level.  We  feel  that 
stopping  at  iteration  136  (four  variables  remain)  is  acceptable  because  it  maintains  a 
prediction  level  of  85.2%.  The  four  remaining  variables  are  ammunition,  fuel,  engine 
spares,  and  maintenance  crews.  These  variables  explain  the  most  amount  of  variability  in 
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the  logistics  simulation  model  and  are  very  promising  to  see.  As  we  discussed  in  Chapter 
2  of  this  paper,  ACS  maintains,  supplies,  and  equips  personnel  and  facilities.  These  ANN 
results  showcase  that  ammunition,  fuel,  engine  spares,  and  maintenance  crews  are  key  in 
capturing  ACS  through  our  selected  MOEs.  These  variables  could,  in  turn,  be  used  to  run 
statistical  analysis  before,  during,  or  after  the  wargame. 

To  ensure  that  the  ANN  is  accurate  in  predicting  the  class,  we  develop  a 
confusion  matrix  as  shown  in  Figure  13  where  Class  1  is  SC  and  Class  2  is  SNC. 
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Figure  13.  ANN  Confusion  Matrix 


Figure  13  shows  that  the  training  set  was  able  to  obtain  an  overall  prediction  level 


of  85.5%,  validation  set  has  an  82.3%  overall  prediction  level,  testing  set  has  an  89.2% 


overall  prediction  level  and  the  entire  set  achieves  an  overall  prediction  level  of  85.6%. 


These  prediction  levels  all  validate  that  the  model  is  working  effectively.  The  levels  are 
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all  within  a  reasonable  distance  of  each  other.  In  addition,  this  overall  prediction  level  of 
85.6%  far  exceeds  our  minimum  requirement  of  obtaining  a  75%  prediction  level. 

As  we  stated  previously,  our  data  set  is  unbalanced  in  terms  of  our  response 
variable.  We  didn’t  have  a  50/50  split  between  SC  and  SNC.  The  result  of  this 
unbalanced  set  is  that  the  ANN  models  make  biased  decisions  towards  the  majority  class 
(Ganganwar,  2012: 1).  In  other  words,  the  ANN  models  constructed  for  this  paper 
gravitate  towards  predicting  SC  and  cause  the  prediction  levels  to  be  skewed.  The 
amount  of  skewness  was  not  detennined  for  our  study.  A  reasonable  assumption  for  the 
potential  effect  based  upon  the  literature  is  on  the  order  of  5%  for  each  class.  Further 
analysis  could  be  completed  to  detennine  appropriate  measures  to  address  this  issue; 
however,  this  was  beyond  the  scope  of  our  study.  Some  alternative  approaches  are 
oversampling,  undersampling,  adjusting  the  weights  assigned  to  the  classes,  adjusting  the 
decision  threshold,  the  snowball  method,  and/or  the  k  nearest  neighbor  (kNN)  method 
(Ganganwar,  2012:2). 

ANN  is  able  to  reduce  the  model  from  140  variables  to  4  variables  with  a 
prediction  level  of  approximately  85%.  The  remaining  variables  are  ammunition,  fuel, 
engine  spares  and  maintenance  crews  which  are  able  to  effectively  capture  ACS.  These 
variables  could  be  used  to  showcase  to  commanders  what  the  limitations  and  constraints 
of  ACS  are  within  wargames.  In  addition,  these  variables  are  the  metrics  that  would  be 
used  to  develop  a  metamodel  that  has  faster  run  times  and  the  capability  to  provide  real 
time  logistic  impact  during  wargames. 
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Summary 

We  provide  a  framework  of  the  analysis  that  could  be  conducted  before  a 
wargame  begins.  We  use  statistical  analysis  with  plots,  paired  t-tests,  and  confidence 
intervals  to  show  that  Base  C  was  perfonning  better  than  Base  A  in  JSF  operational  ready 
rates  and  sortie  completion  rates.  These  two  insights  are  then  able  to  provide  Blue 
commanders  with  the  more  realistic  capabilities  of  their  force.  Our  second  analysis  used 
multivariate  analysis  to  reduce  the  number  of  variables  and  develop  a  metamodel.  We 
successfully  completed  these  two  goals  by  using  ANN  and  showcase  that  ammunition, 
fuel,  engine  spares,  and  maintenance  crew  metrics  are  key  in  capturing  ACS  through  our 
selected  MOEs.  This  is  a  very  important  finding  because  we  can  use  these  metrics  not 
only  in  the  metamodel,  but  also  in  statistical  analysis.  With  both  of  these  analyses,  we 
are  able  to  demonstrate  through  a  proof  of  concept  that  a  logistics  simulation  can  and 
should  be  incorporated  into  future  wargames  because  they  provide  more  realistic  force 
capabilities  by  capturing  ACS  constraints  and  limitations. 
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V.  Conclusions  and  Recommendations 


The  purpose  of  this  thesis  is  to  gain  insights  on  the  incorporation  of  logistics 
within  wargames.  Including  logistics  provides  wargames  with  more  accurate 
representations  of  Blue  Forces,  and  strengthens  the  insights  gathered  from  wargames. 
Combining  these  two  aspects  allows  for  considerations  of  development,  deployment, 
employment,  and  procurement  of  future  weapon  systems  within  the  DoD. 

Conclusions  of  Research 

There  have  been  numerous  reasons  why  logistics  have  failed  to  be  incorporated 
within  Title  10  wargames.  Wargames  have  been  designed  to  test  anned  strategies  against 
opposing  forces  over  a  short  time  window.  Under  this  design,  it  has  also  been  assumed 
that  logistics  can  support  any  reasonable  set  of  operations  conducted  during  a  wargame. 
As  shown  in  Chapter  4,  including  logistics  in  a  wargame  provides  commanders  with  a 
more  accurate  idea  of  forces  available  for  combat  operations.  There  are  issues  involved 
with  maintenance,  supply,  and  manpower  that  limit  the  number  of  combat  sorties  that  can 
be  completed  on  a  daily  basis.  The  bottom  line  is  that  incorporating  logistics  into 
wargames  provide  more  realistic  representation  of  Blue  Force  capabilities. 

In  the  recent  past,  a  separate  logistics  simulation  has  been  modified  and 
completed  at  the  conclusion  of  a  wargame.  There  has  been  minimal  success  with  this 
approach  because  feasibility  results  are  provided  several  days  after  the  wargame  is 
completed.  This  approach  doesn’t  allow  Blue  Force  commanders  to  alter  their  approach 
and  retest  their  updated  strategies  in  the  wargame  scenario;  however,  the  results  have 
shown  interesting  results  with  omitting  logistics  from  wargames.  These  results  have 
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brought  greater  attention  to  the  incorporation  of  logistics  within  wargames  because  they 
have  showcased  that  the  assumption  that  logistics  can  support  all  operations  during 
wargames  is  false.  While  these  are  great  results,  this  still  leave  the  fundamental  issue 
with  completing  the  separate  logistics  simulation  model  after  the  wargame  has  concluded. 

The  most  ideal  approach  would  be  to  modify  existing  wargame  combat  models 
and  include  logistics  within  them.  As  we  discussed  in  Chapter  2  of  this  paper,  this  is  not 
an  easy  task  because  of  the  conflicting  aggregation  levels  between  logistic  simulations 
and  combat  models.  This  conflict  comes  down  to  a  difference  in  metrics  and  processes 
that  might  not  have  the  ability  to  communicate  with  each  other  within  a  single  simulation 
platform.  The  ability  to  provide  instantaneous  results  to  commanders  is  an  immense 
benefit  though.  It  makes  the  wargame  more  realistic  because  it  forces  commanders  to 
provide  near  real-time  decisions,  and  in  turn,  provide  better  insights.  As  we  discussed  in 
Chapter  3,  this  still  leaves  the  issue  of  long  run  times.  Constructing  a  single  wargame 
model  that  captures  attrition,  decision  making,  and  logistics  is  not  feasible  due  to  these 
and  other  reasons. 

The  most  appropriate  approach  would  be  to  utilize  a  logistics  simulation  prior  to  a 
wargame.  This  allows  for  ACS  to  provide  further  representation  of  Blue  Forces  and 
provide  more  impactful  insights  from  wargames  by  allowing  commanders  to  alter  their 
strategies  during  the  wargame.  In  addition,  a  number  of  excursions  could  be  run  with  the 
logistics  simulation  model  to  mimic  possible  decisions  made  by  commanders  during 
wargames.  Ultimately,  completing  the  logistics  simulation  model  before  a  wargame 
provides  plenty  of  time  to  modify,  complete  and  analyze  the  separate  logistics  model. 
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We’ve  discussed  at  what  point  in  the  process  to  run  a  logistics  simulation,  but  we 
needed  to  also  determine  how  we  could  incorporate  ACS  within  Title  10  Wargames.  We 
felt  that  using  a  proof  of  concept  with  a  separate  logistics  simulation  model  incorporated 
into  a  wargame  scenario  showcased  this.  It  demonstrated  that  the  setup,  run  times  and 
analysis  could  all  be  completed  within  a  reasonable  amount  of  time.  It  also  showed  that  a 
logistics  model  could  be  tied  in  with  a  wargame  by  providing  further  interpretations  of 
the  capabilities  of  Blue  Forces  through  analysis  of  key  ACS  MOEs. 

We  selected  to  use  LCOM-ATK  as  our  platform  because  it  allows  its  users  full 
control  over  the  model  constructed  and  captures  the  missions  that  ACS  performs.  We 
successfully  developed  a  fictional  wargame  scenario  in  the  Asia-Pacific  region  that  has 
three  Blue  FOBs  conducting  operations  over  two  Red  ground  areas  of  control  over  a  180 
day  period.  The  Blue  offensive  forces  were  composed  of  a  squadron  of  JSF  per  FOB,  but 
the  main  component  of  the  model  was  the  supply  and  maintenance  of  the  JSF.  Wargames 
have  used  aggregated  forces  such  as  a  full  squadron  of  JSF  to  be  used  against  enemy 
forces,  but  we  want  to  show  that  ACS  can  only  operate  and  maintain  a  portion  of  that 
squadron  a  day. 

The  analysis  of  the  output  of  the  logistics  simulation  was  partitioned  into  two 
cases.  The  first  case  focused  on  analyzing  the  data  prior  to  the  wargame.  We  used  plots, 
95%  confidence  intervals,  and  hypothesis  tests  to  provide  Blue  Force  commanders  with 
more  realistic  representations  of  their  forces.  Our  analysis  showed  that  the  average 
number  of  sorties  that  could  be  completed  per  day  was  8  sorties,  and  that  ACS  could 
operate  and  maintain  30%  of  the  JSF  squadron.  This  is  a  key  insight  because  wargame 
commanders  should  recognize  some  level  of  degraded  operational  capability.  Insights 
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like  this  drive  many  factors  within  the  DoD.  This  forces  decision  makers  to  detennine 
matters  such  as  procuring  more  units  of  an  aircraft  and/or  funding  for  more  personnel  in 
AFMC  to  operate  and  maintain  a  larger  amount  of  sorties. 

The  second  case  for  our  analysis  was  to  complete  Multivariate  Analysis 
techniques  on  the  logistics  output  data.  This  type  of  analysis  was  used  to  detennine  key 
ACS  factors  and  reduce  the  dimensionality  of  logistics  models.  We  used  the  70/15/15 
rule  in  regards  to  analyzing,  testing  and  validating  the  data.  PCA,  FA  and  DA  were  all 
able  to  reduce  the  dimensionality  of  the  logistics  model  but  the  reduction  was  not  enough. 
ANN  was  able  to  produce  a  metamodel  by  reducing  the  dimensionality  of  the  logistics 
model  to  four  variables  while  obtaining  a  prediction  level  of  85%  for  correctly  classifying 
a  sortie  as  being  completed  or  not.  More  importantly,  the  four  variables  that  remained 
were  ammunition,  fuel,  engine  spares,  and  maintenance  crews  that  showcase  ACS  can  be 
captured  and  communicated  to  commanders  using  a  minimal  subset  of  factors. 

Essentially,  these  four  ACS  inputs  can  accurately  predict  force  capabilities.  We  picked 
sorties  completed  (SC)  or  not  (SNC)  as  our  response  variable,  but  we  could  use  this  same 
approach  for  other  factors.  Our  metamodel,  in  this  case  an  ANN,  has  significantly  lower 
run  times  and  the  ability  to  accurately  predict  Blue  Force  capabilities  utilizing  the  four 
ACS  factors.  This  ANN  is  designed  to  be  run  at  the  end  of  a  wargame  move  to  provide 
commanders  a  more  realistic  picture  of  the  force  capabilities  available  for  the  next  move. 

Incorporating  logistics  into  wargames  is  not  an  easy  task.  Approaching  the 
problem  by  capturing  logistics  processes  in  combat  models  is  not  feasible  because  there 
are  too  many  issues  with  conflicting  aggregations  levels  to  proceed  with  this  approach. 

As  we  showcased  in  this  paper,  utilizing  a  verified,  validated  and  accredited  logistics 
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simulation  model  is  the  most  practical  approach.  It  allows  ACS  to  further  describe  Blue 
Forces  by  inputting  their  limitations  and  constraints  within  the  stochastic  logistics 
simulation  model.  Using  various  analytical  techniques,  we  were  able  to  show  that 
ammunition,  fuel,  engine  spares,  and  maintenance  crews  are  very  important  in  describing 
ACS.  These  metrics  have  the  ability  to  predict  and  provide  commanders  with  meaningful 
results.  In  the  end,  wargames  must  evolve  once  again,  and  incorporate  logistics  within 
them.  This  evolution  ensures  commanders  are  given  insights  that  continue  to  maintain 
the  United  States  Air  Force’s  (USAF)  dominance  in  air,  space,  and  cyberspace. 

Recommendations  for  Future  Research 

This  thesis  provides  opportunities  for  future  work  within  wargames  and  logistics. 
The  lessons  learned  throughout  the  process  of  developing  a  proof  of  concept  showcased 
various  areas  that  need  further  development. 

Analyzing/Prolonging  Wargames 

The  individuals  that  have  designed  wargames  throughout  the  years  have 
constructed  them  such  that  they  are  meant  to  test  and  develop  combat  strategies  over  a 
short  time  window.  There  has  been  little,  if  any,  analysis  on  longer  duration  wargames  in 
Unified  Engagements.  With  today’s  ever  increasing  constrained  budget,  wargames  can 
provide  a  tool  in  which  senior  leaders  exercise  and  evaluate  combat  operations  over  a 
longer  period  in  time.  This  would  allow  analysts  to  better  detennine  the  costs  and 
resources  associated  with  a  sustained  engagement.  An  approach  that  could  be  utilized  for 
this  would  be  to  add  extra  time  at  the  end  of  standard  wargames  to  consider  longer  term 
engagement.  This  approach  would  allow  for  further  analysis  and  a  better  representation 
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of  readiness  levels  and  force  capabilities  after  initial  surge.  We  recommend  perfonning 
an  analysis  on  the  methodology  and  length  that  wargames  currently  use. 

When  Logistics  Impacts 

We  arrived  at  a  question  of  when  does  logistics  impact  a  mission.  We  came  to 
our  conclusion  that  180  days  is  sufficient  because  it  allows  for  operations  to  carry  out  and 
for  parts  to  begin  to  fail.  Perfonning  a  model  with  too  few  days  doesn’t  allow  for 
processes  to  breakdown  nor  be  impactful.  We  recommend  perfonning  an  analysis  on 
detennining  what  the  minimum,  average  and  maximum  amount  of  time  required  to 
capture  logistics  within  simulations. 
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Appendix  A:  Measures  of  Effectiveness 
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POST  SORTIE  TIME(HRS) 

B4  AVG  TIME  TO  GET 

RESOURCE(HRS) 

A5  NUMBER  OF 

SORTIES  INITIATED 

A24  NO.  OF 

POST  SORTIES  COMPLETED 

B5  MIN  TIME  TO  GET 

RESOURCE(HRS) 

A6  PERCENT 

INITIATED 

A25  NO.  OF 

PLUGGED  SORTIES 

B6  MAX  TIME  TO  GET 

RESOURCE(HRS) 

A7  NUMBER  OF 

ATTRITIONS 

A26  NUMBER 

OF  SORTIES  COMPLETED 

B7  STD  DEV  TO  GET 

RESOURCE(HRS) 

A8  NUMBER  OF 

RAM  REPAIRS 

A27  AVG 

SORTIE  LENGTH  (HRS) 

B8  AVG.  ACTIVITY 

LENGTH  (HRS) 

A9  #  OF  AIR 

ABORTS 

A28  MIN  SORTIE 

LENGTH  (HRS) 

B9  MIN.  ACTIVITY  LENGTH 

(HRS) 

A10  #  OF 

SYMPATHETIC  AIR  ABORTS 

A29  MAX 

SORTIE  LENGTH  (HRS) 

BIO  MAX.  ACTIVITY 

LENGTH  (HRS) 

All  AVG.  AC 

PRESORTIE  MAINT.(HRS) 

A30  STD  DEV 

SORTIE  LENGTH  (HRS) 

B11  STD  DEV  ACTIVITY 

LENGTH  (HRS) 

A12  MIN 

PRESORTIE  MAINT.(HRS) 

A31  AC  WHICH 

WENT  INTO  MISN  WAIT 

B12  NO.  OF  ACTIVITIES 

COMPLETED 

A13  MAX 

PRESORTIE  MAINT.(HRS) 

A32  #  OF  FWR 

MISSIONS  REQUESTED 

B13  #EXOG  ACTIVITIES 

REQUESTED 

A14  STD  DEV 

PRESORTIE  MAINT.(HRS) 

A33  #  OF  FWR 

MISSIONS  ACCOMPLISHD 

B14  NO.  ACTIVITIES  MIN 

CANCELLED 

A15  NO.  OF  PRE 

SORTIES  COMPLETED 

A34  #  OF  FWR 

SORTIES  REQUESTED 

Cl  NUMBER  OF  AIRCRAFT 

AUTH.(EOP) 

A16  AVG.  AC  MISN 

WAIT  STATUS(HRS) 

A35  #  OF  FWR 

SORTIES  INITIATED 

C2  NUMBER  OF  AIRCRAFT 

DAYS  AVAIL 

A17  MIN  MISN  WAIT 

STATUS(HRS) 

A36  #  OF 

ENGINE  FAILURES 

C3  PCT  SORTIES(INCL 

ALERT) 

A18  MAX  MISN 

WAIT  STATUS(HRS) 

A35  #  OF  FUEL 

C4  PCT  UNSCHED 

MAINTENANCE 

A19  STD  DEV  MISN 

WAIT  STATUS(HRS) 

A35  #  OF 

AMMUNITION 

C5  PCT  SCHED 

MAINTENANCE 
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C6  PCT  NMCS 

C25  %NOT  MISSION 

CAPABLE/NMC  RATE 

D19  DEPRECATED  STAT 

C7  PCT  TIME  WAITING  TO 

FLY 

D1  MANHOURS  AVAILABLE 

D20  NMCM  INDICATOR 

C8  PCT  TIME  WAIT 

RESOURCES 

D2  PERCENT  UTILIZATION 

El  NO.  OF  REP  GENS  OR  EXOG 

DMDS 

C9  PCT  OPERATIONALLY 

READY 

D3  MANHOURS  USED 

E2  PCT  BASE  REPAIR 

CIO  AVG.  AC  POST  SORTIE 

TIME(HRS) 

D4  PCT  USED  -  UNSCHED 

MAINT 

E3  PCT  DEPOT  REPAIR 

C11  MIN.  AC  POST  SORTIE 

TIME(HRS) 

D5  PCT  USED  -  SCHED 

MAINT 

E4  AVG.  BASE  REPAIR  CYCLE 

(DAYS) 

C12  MAX.  AC  POST  SORTIE 

TIME(HRS) 

D6  PCT  USED  AS  A  PRIME 

E5  MIN.  BASE  REPAIR  CYCLE 

(DAYS) 

C13  STD  DEV  POST  SORTIE 

TIME(HRS) 

D7  PCT  USED  AS  A 

SUBSTITUTE 

E6  MAX.  BASE  REPAIR  CYCLE 

(DAYS) 

C14  REQUESTED  SORTIES/ 

AC /DAY 

D8  ON-EQUIP  MAN-HOURS 

USED 

E7  STD  DEV  BASE  REPAIR  CYCLE 

C15  ACHIEVED  SORTIES/ AC 

/DAY 

D9  OFF-EQUIP  MAN¬ 

HOURS  USED 

E8  PCT  TIME  ACTIVE  REPAIR 

C16  FLYING  HOURS 

DIO  NUMBER  OF  MEN 

DEMANDED 

E9  PCT  TIME  WAIT  RESOURCES 

C17  AVG.  FLYING  HOURS/ 

AC /DAY 

Dll  NO.  OF  MEN 

DEMANDED  POST  SCAN 

E10  NO.  OF  ITEMS  IN  REPAIR 

(EOP) 

C18  AVG.  AC  PRE  SORTIE 

TIME  (HRS) 

D12  PCT  PROV.  BY 

ONHAND  BAL. 

Ell  NO.  OF  ITEMS  BACKLOGGED 

(EOP) 

C19  MIN.  AC  PRE  SORTIE 

TIME  (HRS) 

D13  PCT  PROV.  BY  GEN 

SUBS 

FI  AUTHORIZED  QUANTITY 

C20  MAX.  AC  PRE  SORTIE 

TIME  (HRS) 

D14  PCT  PROV.  BY 

EXPEDITE 

F2  PCT  PROV.  BY  PRIME  ONHAND 

BAL 

C21  STD  DEV  PRE  SORTIE 

TIME  (HRS) 

D15  PCT  PROV.  BY 

PREEMPTION 

F3  NUMBER  OF  BACKORDER-DAYS 

C22  NO.  OF  PRE  SORTIES 

COMPLETED 

D16  PCT  DEMANDS  NOT 

SATIS. 

F4  NUMBER  OF  UNITS  DEMANDED 

C23  NO.  OF  POST  SORTIES 

COMPLETED 

D17  OVERTIME 

MANHOURS  USED 

F5  PCT  PROV.  PRIME  OR  SUBS 

C24  %MISSION  CAPABLE/MC 

RATE 

D18  SIMULATED  MH  PER 

FLYING  HOUR 

F6  PCT  PROV.  BY  EXPEDITE 

61 


F7 

PREEMPTION 

PCT  PROV.  BY 

G13 

SUBS 

PCT  PROV.  BY  GEN 

F8 

SATIS. 

PCT  DEMANDS  NOT 

G14 

EXPEDITE 

PCT  PROV.  BY 

F9  NUMBER  OF 

CANNIBALIZATIONS 

G15 

PREEMPTION 

PCT  PROV.  BY 

F10  NO.  ITEMS  ON 

BACKORDER  (EOP) 

G16 

SATIS. 

PCT  DEMANDS  NOT 

F11 

NMCS  INDICATOR 

G17 

BACKLOG  (EOP) 

EQUIP  HOURS 

F12 

NUMBER  OF  ORDERS 

G18 

GENERATED 

NUMBER  OF  UNITS 

F13 

INS 

NUMBER  OF  TURN- 

G19  NO.  ITEMS  ON 

BACKORDER  (EOP) 

G1 

QUANTITY 

AUTHORIZED 

H6 

USED 

FACILITY  HOURS 

G2 

AVAIL. 

EQUIPMENT  HOURS 

H9 

DAYS 

#  OF  BACKORDER 

G3 

MAI  NT 

PCT  USED  -UNSCHED 

H16 

SATISFIED 

%  DEMANDS  NOT 

G4 

MAINT 

PCT  USED  -SCHED 

H17 

USED/DEMAND 

AVERAGE  HOURS 

G5 

PCT  AVAIL  -  UNUSED 

H19 

BACKLOG  (EOP) 

FACILITY  HOURS 

G6 

USED 

EQUIPMENT  HOURS 

G7 

PCT  USED  AS  A  PRIME 

G8 

SUBSTITUTE 

PCT  USED  AS  A 

G9  NUMBER  OF 

BACKORDER-DAYS 

G10 

DEMANDED 

NUMBER  OF  UNITS 

G11  NO.  UNITS 

DEMANDED  POST  SCAN 

G12 

ONHAND  BAL 

PCT  PROV.  BY 
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Appendix  B:  Multivariate  Analysis  Code 


%PCA  &  FA  Train  Final 
clc 

clear  all 
close  all 

%load  data 

PCATrain  =  xlsread ( ' test . xls ' ) ; 
%Indicator  variable 
load  result. mat 


%Correlation  matrix 

cMat  =  corr (PCATrain (1:738,1:41) ) ; 


%Eigenvalues  and  Eigenvectors 
[V, D]  =  eig(cMat); 

%Sort  eigenvalues 
D  =  sort (diag (D)  ,  'descend'); 


%Horn's  curve 

%  horn  =  xlsread (' FinalHorns . xls ') ; 
%  figure 
%  hold  on 

%  x  =  linspace (0, 10, 40) ; 

%  plot (x, horn, ' b ', x, D, ': r* ' ) 

%  title ('Horns  Test') 

%  legend ( ' Horns ' , ' Eigenvalues ' ) 


%Variance 
OneVec  = 
TwoVec  = 
cumTwo  = 
ThrVec  = 
cumThr  = 
FourVec  = 
cumFour  = 
FiveVec  = 
cumFive  = 
SixVec  = 
cumSix  = 
SevVec  = 
cumSev  = 
SevVec; 
EigVec  = 
cumEig  = 

+  EigVec; 
NineVec  = 
cumNin  = 

+  EigVec 
TenVec  = 


Explained 
D ( 1 , 1 ) /sum (D); 

D (2 , 1 ) /sum (D) ; 

OneVec  +  TwoVec; 

D ( 3 , 1 ) /sum (D) ; 

OneVec  +  TwoVec  +  ThrVec; 


D ( 4  ,  1 ) /sum (D) ; 

OneVec  +  TwoVec  +  ThrVec  +  FourVec; 

D ( 5 , 1 ) /sum (D) ; 

OneVec  +  TwoVec  +  ThrVec  +  FourVec  +  FiveVec 
D ( 6 , 1 ) /sum (D) ; 

OneVec  +  TwoVec  +  ThrVec  +  FourVec  +  FiveVec 
D ( 7 , 1 ) /sum (D) ; 

OneVec  +  TwoVec  +  ThrVec  +  FourVec  +  FiveVec 


D ( 8 , 1 ) /sum (D) ; 

OneVec  +  TwoVec  +  ThrVec  +  FourVec  +  FiveVec 
D ( 9 ,  1 ) /sum (D) ; 

OneVec  +  TwoVec  +  ThrVec  +  FourVec  +  FiveVec 
+  NineVec; 

D ( 10 , 1 ) /sum (D) ; 


+ 

+ 


+ 


+ 


SixVec; 

SixVec 


SixVec 


SixVec 


SevVec 


SevVec 
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cumTen  =  OneVec  +  TwoVec  +  ThrVec  +  FourVec  +  FiveVec  +  SixVec  +  SevVec 
+  EigVec  +  NineVec  +  TenVec; 

%Training  Scores 

stdMat  =  zscore (PCATrain (1 : 738, 1 :  41) ) ; 

%Load  matrix 

PCALoadMat  =  V*sqrt (diag (D) ) ; 

FALoadmat  =  rotatef actors (PCALoadMat (:, 1 : 10) ) ; 

%Scores  matrix 

FAscoresMat  =  stdMat  *  inv(cMat)  *  FALoadmat; 

PCAscoresMat  =  stdMat  *  inv(cMat)  *  PCALoadMat (:, 1 : 10 ) ; 

%Validation  Scores 

stdValMat  =  zscore (PCATrain (739 : 1055, 1 :  41) ) ; 

%Load  matrix 

PCALoadMat  =  V*sqrt (diag (D) ) ; 

FALoadmat  =  rotatef actors (PCALoadMat (:, 1 : 10) ) ; 

%Scores  matrix 

FAValdscoresMat  =  stdValMat  *  inv(cMat)  *  FALoadmat; 

PCAValdscoresMat  =  stdValMat  *  inv(cMat)  *  PCALoadMat (:, 1 : 10 ) ; 

%Kaiser  index 

[ifs2]=  IFS2 (FALoadmat) ; 

ifsFA=sqrt (ifs2) 

[if s2 ] =  IFS2 (PCALoadMat) ; 
ifsPCA=sqrt (ifs2) 

%PCA  Training  Plots 
for  i=l : 10 
for  j=i+l : 10 
figure 

gs cat ter ( PCAscoresMat ( : , i ) , PCAscoresMat ( : , j ) , result ( 1 : 738 , 1)  ,  'br',  'x.') 
legend ( ' SC ' , ' SNC ' ) 

title (' Principal  Component  Analysis') 
strName=sprintf ( ' PC  %d ' , i ) ; 
xlabel (strName) 
strName2=sprintf ( ' PC  %g ' , j ) ; 
ylabel (strName2) 

end 

end 

%FA  Training  Plots 
for  i=l : 10 
for  j=i+l : 10 
figure 

gs cat ter (FAscoresMat ( : , i ) , FAscoresMat (:,j),result(l:738,l), 'br', 'x. ') 
legend ( 'SC' , 'SNC' ) 
title (' Factor  Analysis') 
strName=sprintf (' Factor  %d',i); 
xlabel (strName) 

strName2=sprintf ( ' Factor  %g ' , j ) ; 
ylabel (strName2) 
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end 

end 


%PCA  Validation  Plots 
for  i=l : 10 
for  j=i+l : 10 
figure 

gscatter ( PCAValdscoresMat ( : , i) , PCAValdscoresMat ( : , j ) , result (739 : 1055, 1) 
, 'br ' , 'x. ' ) 

legend ( ' SC ' , ' SNC ' ) 

title (' Principal  Component  Analysis  Validation') 

strName=sprintf ( ' PC  %d ' , i ) ; 

xlabel (strName) 

strName2=sprintf ( ' PC  %g ' , j ) ; 

ylabel (strName2) 

end 

end 

%  FA  Validation  Plots 
for  i=l : 10 
for  j=i+l : 10 
figure 

gscatter (FAValdscoresMat ( : , i) , FAValdscoresMat ( : , j ) , result (739 : 1055, 1)  , 

br ' , 'x. ' ) 

legend ( ' SC ' , ' SNC ' ) 

title (' Factor  Analysis  Validation') 
strName=sprintf (' Factor  %d',i); 
xlabel (strName) 

strName2=sprintf ( ' Factor  %g ' , j ) ; 
ylabel (strName2) 

end 

end 

%DA 

clc 

clear  all 
close  all 

%load  data 

DATrain  =  xlsread (' DATrain . xls ') ; 

LCOMData  =  DATrain; 
outMat  =  LCOMData  (:, 1 :  41) ; 

%1 

SC=LCOMData (1 : 1254, 1 : 141) ; 

SNC=LCOMData (255  rend, 1:141); 
cov_SC=cov (SC)  ; 
cov_SNC=cov (SNC) ; 
mean_SC=mean (SC) ; 
mean  SNC=mean (SNC) ; 
prior_SC=254/17  652  ; 
prior  SNC=498/17652; 
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diq_SC=zeros  (752,1)  ; 
diq_SNC=zeros  (752,1); 
y=zeros  (752,1) ; 
for  i  =  1:17652 

diq_SC (i)=-.5*log(det (cov_SC) ) - . 5* (outMat ( i , 1 : 41 ) - 
mean_SC) *inv (cov_SC) * (outMat (i, 1 : 41) -mean_SC) ' +log (prior_SC) ; 
diq_SNC (i)=-.5*log(det (cov_SNC) ) - . 5* (outMat (i,  1:41)  — 
mean  SNC) *inv (cov_SNC) * (outMat (i, 1 : 41) -mean_SNC) ' +log (prior  SNC) ; 
if  diq_SC ( i ) >diq_SNC ( i ) 
y (i) =1; 

end 

end 

class= [diq_SC  diq_SNC  y]  ; 
correct l=sum (y ( 1 : 12  654 ) ) ; 
wrong2=sum (y (255 : 17  652 ) )  ; 

confusion  l=[correctl  12 65-correctl ; wrong2  17862-wrong2] 

test_with=zeros (2,1)  ; 
test_without=zeros (2,1)  ; 
test_result=zeros (2,1)  ; 
for  i  =  1 : 2 

test_with (i)=-.5*log(det (cov_SC) ) - . 5* (outMat ( i , 1 : 14 1 ) - 
mean_SC) *inv (cov_SC) * (outMat (i, 1 : 141) -mean_SC) ' flog (prior_SC) ; 

test_without (i)=-.5*log(det (cov_SNC) ) - . 5* (outMat ( i , 1 : 14 1 ) - 
mean  SNC) *inv (cov_SNC) * (outMat (i, 1 : 141) -mean_SNC) ' tlog (prior  SNC) 

if  test_with ( i ) >test_without ( i ) 
test_result ( i ) =1 ; 

end 

end 

[test_with  test_without  test_result] 

temp=corr ( [LCOMData ( : , 1 : 141)  max (diq_SC, diq_SNC) ] ) ; 
loadingsl=temp (end, 1 : 141) ' 

[~, ind] =min (abs (loadingsl) ) ; 

LCOMData (:,ind)=[] ; 

GM ( : , ind) = [ ] ; 

%ANN 

%load  data 

LCOM  =  xlsread ( ' test . xls ' ) 

post  =1; 

if  post  ==  0 

%noise  generator 
x  =  rand (17642, 1) ; 

%create  data  matrix 
temp  =  [x  LCOM (:, 1 : 141) ] ; 
data  =  temp ( : , [ 1  2 ] ) ; 
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%std  data 

input  =  zscore (data) ; 

%create  output  matrix 

ouput  =  [LCOM ( : , 142) , 1  -  LCOM ( : , 142 ) ] ; 

else 

snr=  [  ] ; 

wts=net . IW{ 1 , 1 } ; 
dim=size (wts) ; 
noise=wts ( : , 1 ) ' *wts (  :  ,  1 )  ; 
for  j  =2 : dim (2 ) 

snr ( j ) =10*logl0 ( (wts ( : , j ) ' *wts ( : , j ) ) /noise) 

end 


end 

%ANN  NPRScript 
clear  all 
close  all 
count=l ; 

load ( ' cleandata ' ) 
load ( ' label_pro j '  ) 
data=cleandata; 
output=data  ( : ,  1 )  ; 
data (  :  ,  1)  =  [  ]  ; 
input=zscore (data) ; 
permmat= [ ] ; 

while  count  <  2 

inputs  =  input'; 
targets  =  output'; 

%  Create  a  Pattern  Recognition  Network 

hiddenLayerSize  =  141; 

net  =  patternnet (hiddenLayerSize); 


%  Setup  Division  of  Data  for  Training,  Validation,  Testing 
net . divideParam. trainRatio  =  70/100; 
net . divideParam. valRatio  =  15/100; 
net . divideParam. testRatio  =  15/100; 
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%  Train  the  Network 

[net,tr]  =  train (net, inputs, targets) ; 

%  Test  the  Network 

outputs  =  net (inputs); 

errors  =  gsubtract (targets, outputs) ; 

performance  =  perform (net, targets, outputs) 

permmat= [permmat  performance] ; 

figure 

plotconfusion (targets, outputs) 
trainTargets  =  targets  .*  tr . trainMask] 1 } ; 
valTargets  =  targets  .*  tr . valMask] 1 } ; 
testTargets  =  targets  . *  tr . testMask] 1 } ; 
trainPerf ormance  =  perform (net, trainTargets, outputs) 
valPerf ormance  =  perform (net, valTargets, outputs) 
testPerf ormance  =  perform (net, testTargets, outputs) 


snr=  [  ] ; 

wts=net . IW{ 1 , 1 } ; 
dim=size (wts) ; 
noise=wts ( : , 1 )  ' *wts  (  : , 1 ) ; 
for  j=2 : dim (2 ) 

snr ( j ) =10*logl0 ( (wts ( : , j ) ' *wts ( : , j ) ) /noise) ; 

end 

snr 

[val  ind]=min(snr(2:end)); 
minind=ind+l 
input ( : , minind)  =  [ ]  ; 
count=count+l ; 
end 
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14.  ABSTRACT 

Air  Force  (AF)  Concept  Development  and  Experimentation  (CD&E)  continually  progresses  the  evolution  of  AF  while  achieving 
national  security  and  military  objectives.  CD&E  experiments  on  future  challenges,  procurement  of  new  weapon  systems  and  tests 
existing/innovative  strategies  as  potential  solutions.  The  primary  tool  that  CD&E  utilizes  in  conducting  these  experiments  is 
wargaming.  This  thesis  provides  a  foundation  to  incorporate  logistics  into  Air  Force  Title  10  wargames.  More  specifically,  we 
capture  Air  Force  Materiel  Command’s  (AFMC)  Agile  Combat  Support  (ACS)  within  an  unclassified  general  wargame  scenario. 
Logistics  has  been  omitted  from  wargames  for  a  multitude  of  reasons  throughout  the  years.  We  develop  a  logistics  simulation  model 
of  a  simplified  wargame  scenario  designed  to  be  run  within  the  Logistics  Composite  Model  (LCOM)  Analysis  Toolkit  (ATK) 
version  4.0  before  a  wargame  initiates.  We  capture  ACS  within  the  stochastic  simulation  by  incorporating  engine  failures, 
maintenance  crews,  ammunition,  fuel,  and  various  other  logistics  metrics.  By  varying  the  types  of  sortie  operations  and  the  logistics 
support  available,  further  insight  is  gathered  on  Blue  Force  capabilities.  We  develop  decision  quality  information  to  present  to  a 
decision  maker  by  combining  statistical  and  multivariate  analysis.  Our  approach  showcases  how  to  gather  insights  from  ACS 
metrics,  including  development  of  a  metamodel  using  only  four  metrics  to  successfully  predict  key  ACS  Measures  of  Effectiveness 
(MOEs).  Ultimately,  we  design,  analyze  and  demonstrate  that  logistics  can  and  should  be  incorporated  into  wargames. 
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